08:41:04,585 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:41:05,147 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [5078368300866290827] 08:41:05,166 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:41:05,340 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:41:05,340 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:41:05,367 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:05,371 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:05,774 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:41:05,775 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:41:05,790 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:05,795 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:05,801 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:41:06,699 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testBackupDatastore 08:41:06,767 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] 08:41:06,812 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:06,832 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:41:06,833 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:41:06,833 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:41:06,841 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testBackupDatastore 08:41:06,893 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testBackupDatastore 08:41:06,919 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:06,921 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 08:41:06,921 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 08:41:06,960 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 08:41:07,20 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@34066792} 08:41:07,49 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} started 08:41:07,55 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:07,55 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testBackupDatastore 08:41:07,59 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 08:41:07,59 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 08:41:07,60 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore on shard member-1-shard-cars-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#1754774040] 08:41:07,60 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testBackupDatastore 08:41:07,63 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 08:41:07,64 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 08:41:07,64 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore on shard member-1-shard-people-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-976050999] 08:41:07,65 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#1754774040] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:07,66 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-976050999] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:07,116 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] 08:41:07,118 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:07,119 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:41:07,146 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Shard created, persistent : true 08:41:07,146 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Shard created, persistent : true 08:41:07,170 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] 08:41:07,171 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:07,171 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:41:07,221 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] 08:41:07,222 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:07,222 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:41:07,270 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#1594026690 created and ready for shard:member-1-shard-cars-config_testBackupDatastore 08:41:07,270 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#150753661 created and ready for shard:member-1-shard-people-config_testBackupDatastore 08:41:07,271 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#1594026690] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:07,271 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#150753661] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:07,272 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] 08:41:07,273 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:07,273 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:41:07,277 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-people-config_testBackupDatastore: Starting recovery with journal batch size 1 08:41:07,277 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-cars-config_testBackupDatastore: Starting recovery with journal batch size 1 08:41:07,323 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] 08:41:07,324 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:07,324 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:41:07,374 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] 08:41:07,375 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:07,375 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:41:07,425 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] 08:41:07,425 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:07,426 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:41:07,460 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-config_testBackupDatastore: journal open: applyTo=0 08:41:07,460 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testBackupDatastore: journal open: applyTo=0 08:41:07,476 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] 08:41:07,476 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:07,477 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:41:07,499 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-cars-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:07,499 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:07,503 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-cars-config_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 08:41:07,504 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-config_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 08:41:07,511 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:41:07,511 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testBackupDatastore is initialized 08:41:07,512 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:41:07,512 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testBackupDatastore is initialized 08:41:07,516 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:41:07,517 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:07,520 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from null to Follower 08:41:07,521 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:41:07,521 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:07,521 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from null to Follower 08:41:07,527 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] 08:41:07,527 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:07,528 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#1754774040] 08:41:07,542 AM [cluster-test-shard-dispatcher-10] [INFO] Candidate - member-1-shard-cars-config_testBackupDatastore (Candidate): Starting new election term 1 08:41:07,543 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-cars-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:07,543 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:41:07,543 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:07,543 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Follower to Candidate 08:41:07,545 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-config_testBackupDatastore (Candidate): Starting new election term 1 08:41:07,545 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:07,545 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:41:07,545 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:07,545 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Follower to Candidate 08:41:07,546 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-cars-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:07,546 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 08:41:07,547 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:07,547 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 08:41:07,552 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: became leader with frontend state for [] 08:41:07,552 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: became leader with frontend state for [] 08:41:07,552 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Candidate to Leader 08:41:07,553 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Candidate to Leader 08:41:07,564 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 08:41:07,564 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 08:41:07,565 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@57014553 08:41:07,565 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testBackupDatastore from Candidate to Leader 08:41:07,569 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@69809ce6 08:41:07,569 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testBackupDatastore from Candidate to Leader 08:41:07,569 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testBackupDatastore is ready 08:41:07,616 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] 08:41:07,616 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:41:07,616 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-976050999] 08:41:07,618 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:07,627 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:07,634 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testBackupDatastore 08:41:07,635 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#249673215] 08:41:07,636 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#249673215] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:07,636 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:41:07,638 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:41:07,638 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:41:07,638 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testBackupDatastore 08:41:07,638 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testBackupDatastore 08:41:07,638 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:07,639 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 08:41:07,639 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 08:41:07,642 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 08:41:07,644 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} started 08:41:07,644 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@adfde46} 08:41:07,644 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:07,644 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testBackupDatastore 08:41:07,645 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 08:41:07,645 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 08:41:07,645 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore on shard member-1-shard-cars-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-418179297] 08:41:07,645 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testBackupDatastore 08:41:07,646 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 08:41:07,646 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 08:41:07,646 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore on shard member-1-shard-people-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#1724367669] 08:41:07,647 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Shard created, persistent : true 08:41:07,648 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Shard created, persistent : true 08:41:07,648 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-people-oper_testBackupDatastore: Starting recovery with journal batch size 1 08:41:07,649 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#308822006 created and ready for shard:member-1-shard-people-oper_testBackupDatastore 08:41:07,649 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#-1908889334 created and ready for shard:member-1-shard-cars-oper_testBackupDatastore 08:41:07,650 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testBackupDatastore: journal open: applyTo=0 08:41:07,650 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-418179297] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:07,652 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:07,652 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-cars-oper_testBackupDatastore: Starting recovery with journal batch size 1 08:41:07,654 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-oper_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 08:41:07,655 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testBackupDatastore: journal open: applyTo=0 08:41:07,655 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:41:07,655 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:07,656 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:41:07,656 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testBackupDatastore is initialized 08:41:07,657 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from null to Follower 08:41:07,658 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:07,659 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#1724367669] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:07,659 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#308822006] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:07,659 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-oper_testBackupDatastore (Candidate): Starting new election term 1 08:41:07,660 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:07,660 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:41:07,660 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:07,660 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Follower to Candidate 08:41:07,660 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:07,660 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 08:41:07,660 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: became leader with frontend state for [] 08:41:07,661 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-oper_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 08:41:07,661 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#-1908889334] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:07,660 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Candidate to Leader 08:41:07,662 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:41:07,661 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:41:07,662 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:07,662 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testBackupDatastore is initialized 08:41:07,662 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from null to Follower 08:41:07,663 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 08:41:07,663 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@577d7a80 08:41:07,664 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testBackupDatastore from Candidate to Leader 08:41:07,665 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testBackupDatastore (Candidate): Starting new election term 1 08:41:07,665 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:07,665 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:41:07,665 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:07,666 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:07,666 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 08:41:07,666 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: became leader with frontend state for [] 08:41:07,666 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Follower to Candidate 08:41:07,666 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Candidate to Leader 08:41:07,666 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 08:41:07,666 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4273534e 08:41:07,667 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testBackupDatastore from Candidate to Leader 08:41:07,667 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testBackupDatastore is ready 08:41:07,694 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#249673215] 08:41:07,695 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:41:07,695 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-418179297] 08:41:07,746 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#249673215] 08:41:07,746 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:41:07,746 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#1724367669] 08:41:07,781 AM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 08:41:07,782 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 08:41:07,783 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 08:41:07,788 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 08:41:07,788 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 08:41:07,804 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-cars-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7d2dd153, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testBackupDatastore, ServerConfigPayload=null]] 08:41:07,805 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5ec34a6a, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 08:41:07,816 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@39f0ba91, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 08:41:07,816 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: All shard snapshots received 08:41:07,816 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@8380682, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 08:41:07,817 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 08:41:07,872 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] ClusterAdminRpcService - Successfully backed up datastore to file target/testBackupDatastore 08:41:07,886 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#1759752634] 08:41:07,886 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Received unhandled message org.opendaylight.controller.cluster.datastore.DatastoreContext@3afc1b0e 08:41:07,887 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:07,887 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#1754774040] 08:41:07,919 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Stopping Shard 08:41:07,923 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testBackupDatastore: journal closed 08:41:07,930 AM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 08:41:07,930 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 08:41:07,931 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 08:41:07,932 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6d988571, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 08:41:07,932 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@689a1731, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 08:41:07,932 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 08:41:07,933 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 08:41:07,933 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 08:41:07,935 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3da42b00, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 08:41:08,154 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond 08:41:08,158 AM [cluster-test-pekko.actor.default-dispatcher-19] [ERROR] ClusterAdminRpcService - Failed to back up datastore to file target/testBackupDatastore java.util.concurrent.TimeoutException: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerGetSnapshotReplyActor.onReceive(ShardManagerGetSnapshotReplyActor.java:66) at org.apache.pekko.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:341) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:272) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:233) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:245) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 08:41:08,168 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 08:41:08,172 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testBackupDatastore 08:41:08,175 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testBackupDatastore 08:41:08,179 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@33e3d44e 08:41:08,179 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-people-config_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@33e3d44e 08:41:08,180 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:41:08,180 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Stopping Shard 08:41:08,181 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-config_testBackupDatastore: journal closed 08:41:08,186 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:41:08,186 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:41:08,191 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 08:41:08,193 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testBackupDatastore 08:41:08,193 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@3ec565da 08:41:08,193 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@3ec565da 08:41:08,193 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testBackupDatastore 08:41:08,194 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Stopping Shard 08:41:08,194 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testBackupDatastore: journal closed 08:41:08,194 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5f4fbdc3 08:41:08,194 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-people-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5f4fbdc3 08:41:08,195 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:41:08,195 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Stopping Shard 08:41:08,195 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testBackupDatastore: journal closed 08:41:08,196 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:41:08,196 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:41:08,204 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:08,205 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:08,229 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:41:08,279 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:41:08,309 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-1363858935478749841] 08:41:08,311 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:41:08,315 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:41:08,315 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:41:08,318 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:08,318 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:08,345 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:41:08,345 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:41:08,345 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:41:08,351 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:08,354 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:08,358 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 08:41:08,364 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#548621448] 08:41:08,368 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#548621448] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:08,368 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:41:08,369 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:41:08,369 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:41:08,369 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddReplicasForAllShards 08:41:08,369 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testAddReplicasForAllShards 08:41:08,370 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:08,370 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 08:41:08,370 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 08:41:08,383 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 08:41:08,385 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@284ffa8c} 08:41:08,385 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 08:41:08,386 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:08,386 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddReplicasForAllShards 08:41:08,388 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 08:41:08,388 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 08:41:08,388 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards on shard member-1-shard-cars-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#1104158069] 08:41:08,389 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testAddReplicasForAllShards 08:41:08,389 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 08:41:08,390 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 08:41:08,390 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards on shard member-1-shard-people-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1630194518] 08:41:08,391 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Shard created, persistent : true 08:41:08,391 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#1104158069] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:08,392 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#923606801 created and ready for shard:member-1-shard-cars-config_testAddReplicasForAllShards 08:41:08,392 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:41:08,392 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#923606801] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:08,394 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddReplicasForAllShards: journal open: applyTo=0 08:41:08,394 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Shard created, persistent : true 08:41:08,395 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#-2105167766 created and ready for shard:member-1-shard-people-config_testAddReplicasForAllShards 08:41:08,396 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:08,396 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-people-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:41:08,397 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1630194518] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:08,397 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#-2105167766] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:08,398 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:41:08,399 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:08,399 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:08,399 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-config_testAddReplicasForAllShards: journal open: applyTo=0 08:41:08,401 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 08:41:08,401 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:41:08,401 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddReplicasForAllShards is initialized 08:41:08,402 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:08,404 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 08:41:08,404 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:08,404 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:08,404 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:08,404 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Follower to Candidate 08:41:08,404 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:08,404 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:41:08,404 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: became leader with frontend state for [] 08:41:08,405 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Candidate to Leader 08:41:08,405 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 08:41:08,405 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2a066ee8 08:41:08,405 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Candidate to Leader 08:41:08,405 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:41:08,406 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:41:08,406 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testAddReplicasForAllShards is initialized 08:41:08,406 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:08,406 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:08,406 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 08:41:08,409 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 08:41:08,409 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:08,409 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:08,409 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:08,409 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Follower to Candidate 08:41:08,409 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:08,409 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:41:08,409 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: became leader with frontend state for [] 08:41:08,409 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Candidate to Leader 08:41:08,410 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 08:41:08,410 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@71954daa 08:41:08,410 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Candidate to Leader 08:41:08,410 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:41:08,436 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#548621448] 08:41:08,437 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:08,437 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#1104158069] 08:41:08,488 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#548621448] 08:41:08,488 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:41:08,488 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1630194518] 08:41:08,489 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:08,494 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:08,505 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 08:41:08,507 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1607320106] 08:41:08,508 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1607320106] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:08,509 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:41:08,509 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:41:08,509 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:41:08,509 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddReplicasForAllShards 08:41:08,510 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testAddReplicasForAllShards 08:41:08,510 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:08,510 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 08:41:08,510 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 08:41:08,513 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 08:41:08,514 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@3497ddfc} 08:41:08,514 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 08:41:08,514 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:08,514 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddReplicasForAllShards 08:41:08,516 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Shard created, persistent : true 08:41:08,516 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-668536975] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:08,516 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#1316372739 created and ready for shard:member-1-shard-cars-oper_testAddReplicasForAllShards 08:41:08,517 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:41:08,517 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#1316372739] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:08,518 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddReplicasForAllShards: journal open: applyTo=0 08:41:08,520 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:08,520 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 08:41:08,520 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 08:41:08,521 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards on shard member-1-shard-cars-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-668536975] 08:41:08,521 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testAddReplicasForAllShards 08:41:08,522 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Shard created, persistent : true 08:41:08,522 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#522097454] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:08,523 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#-701573071 created and ready for shard:member-1-shard-people-oper_testAddReplicasForAllShards 08:41:08,523 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-people-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:41:08,523 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#-701573071] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:08,524 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testAddReplicasForAllShards: journal open: applyTo=0 08:41:08,526 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:41:08,526 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:08,526 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:08,526 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 08:41:08,527 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:08,527 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 08:41:08,527 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 08:41:08,527 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards on shard member-1-shard-people-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#522097454] 08:41:08,528 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:41:08,528 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddReplicasForAllShards is initialized 08:41:08,529 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 08:41:08,529 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:08,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:08,529 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:08,529 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 08:41:08,530 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:08,530 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:41:08,530 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: became leader with frontend state for [] 08:41:08,530 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 08:41:08,530 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-people-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:41:08,530 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 08:41:08,530 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:08,530 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:08,531 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 08:41:08,531 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3409832a 08:41:08,531 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Candidate to Leader 08:41:08,531 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:41:08,531 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testAddReplicasForAllShards is initialized 08:41:08,533 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 08:41:08,533 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:08,533 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:08,533 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:08,533 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 08:41:08,533 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:08,533 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:41:08,534 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: became leader with frontend state for [] 08:41:08,534 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 08:41:08,534 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 08:41:08,534 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7661147e 08:41:08,534 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Candidate to Leader 08:41:08,534 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 08:41:08,564 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1607320106] 08:41:08,565 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:41:08,565 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-668536975] 08:41:08,616 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#1607320106] 08:41:08,616 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:41:08,616 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#522097454] 08:41:08,631 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@7f29108] 08:41:08,639 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 08:41:08,641 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Shard created, persistent : true 08:41:08,642 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-pets-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:41:08,642 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards-notifier#1751487339 created and ready for shard:member-1-shard-pets-config_testAddReplicasForAllShards 08:41:08,644 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testAddReplicasForAllShards: journal open: applyTo=0 08:41:08,647 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:08,649 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-pets-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:41:08,650 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:08,650 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:08,650 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 08:41:08,650 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 08:41:08,650 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testAddReplicasForAllShards is initialized 08:41:08,652 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 08:41:08,652 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:08,652 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:08,652 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:08,652 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Follower to Candidate 08:41:08,652 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:08,652 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:41:08,652 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: became leader with frontend state for [] 08:41:08,653 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Candidate to Leader 08:41:08,653 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 08:41:08,653 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2507a391 08:41:08,653 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Candidate to Leader 08:41:08,653 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:41:08,696 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#548621448] 08:41:08,697 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 08:41:08,697 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#766460547] 08:41:08,712 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:08,738 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-5511948437593515863] 08:41:08,740 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.1.5] ... 08:41:08,747 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:08,749 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:41:08,751 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:08,751 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:08,770 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:08,784 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:08,793 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 08:41:08,802 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1173380323] 08:41:08,802 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1173380323] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:08,803 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:41:08,805 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:41:08,805 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:41:08,813 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 08:41:08,814 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@3760f7ca} 08:41:08,814 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 08:41:08,814 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:08,815 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:08,817 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:08,821 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 08:41:08,822 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#395020930] 08:41:08,822 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#395020930] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:08,823 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:41:08,823 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:41:08,823 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:41:08,827 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 08:41:08,828 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@39a70f56} 08:41:08,828 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 08:41:08,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:09,210 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:41:09,271 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:41:09,274 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:09,274 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:41:09,275 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:09,275 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 08:41:09,337 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:41:09,340 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:09,340 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards 08:41:09,340 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 08:41:09,341 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards 08:41:09,341 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 08:41:09,341 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards 08:41:09,341 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 08:41:09,341 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:41:09,340 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:09,342 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards 08:41:09,342 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 08:41:09,342 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards 08:41:09,342 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 08:41:09,343 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 08:41:09,442 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@4f2df350] 08:41:09,444 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 08:41:09,446 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Shard created, persistent : true 08:41:09,446 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-459594008] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:09,447 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#-634827495 created and ready for shard:member-2-shard-pets-config_testAddReplicasForAllShards 08:41:09,447 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#-634827495] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:09,447 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-pets-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:41:09,448 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testAddReplicasForAllShards: journal open: applyTo=0 08:41:09,449 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=no-leader-ns, moduleName=no-leader-module, shardName=no-leader, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@1cfb863e] 08:41:09,449 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2 doCreateShard: shardId: member-2-shard-no-leader-oper_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 08:41:09,450 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Shard created, persistent : true 08:41:09,451 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#1079244336 created and ready for shard:member-2-shard-no-leader-oper_testAddReplicasForAllShards 08:41:09,453 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:41:09,454 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards#542692832] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:09,454 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#1079244336] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:09,454 AM [main] [INFO] ClusterAdminRpcService - Adding replicas for all shards 08:41:09,454 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-no-leader-oper_testAddReplicasForAllShards: journal open: applyTo=0 08:41:09,454 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testAddReplicasForAllShards 08:41:09,455 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:09,455 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, no-leader] for data store oper_testAddReplicasForAllShards 08:41:09,455 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 08:41:09,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 08:41:09,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=pets] 08:41:09,460 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 08:41:09,461 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-pets-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:41:09,462 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:09,462 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:09,462 AM [cluster-test-shard-dispatcher-24] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 08:41:09,462 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 08:41:09,461 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:09,464 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:09,464 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 08:41:09,465 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 08:41:09,465 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=no-leader] 08:41:09,465 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 08:41:09,466 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 08:41:09,466 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=no-leader, waitUntilReady=true] 08:41:09,466 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for no-leader forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 08:41:09,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:09,466 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 08:41:09,466 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 08:41:09,466 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 08:41:09,467 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 08:41:09,467 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for pets forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 08:41:09,467 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 08:41:09,467 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testAddReplicasForAllShards is initialized 08:41:09,469 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 08:41:09,469 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testAddReplicasForAllShards from null to Follower 08:41:09,469 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:41:09,470 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [no-leader] 08:41:09,470 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is initialized 08:41:09,470 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:09,470 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:09,470 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 08:41:09,470 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , received role change from null to Follower 08:41:09,470 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 08:41:09,471 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-no-leader-oper_testAddReplicasForAllShards from null to Follower 08:41:09,477 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:41:09,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-668536975, primaryVersion=13] 08:41:09,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 08:41:09,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#522097454, primaryVersion=13] 08:41:09,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=no-leader, waitUntilReady=true] 08:41:09,479 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: No shard found for no-leader 08:41:09,480 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:41:09,480 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#1104158069, primaryVersion=13] 08:41:09,480 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 08:41:09,480 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1630194518, primaryVersion=13] 08:41:09,481 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=pets, waitUntilReady=true] 08:41:09,481 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#766460547, primaryVersion=13] 08:41:09,495 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Failed to find leader for shard no-leader: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1010) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleReceive(ShardManager.java:243) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:272) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:233) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:245) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 08:41:09,496 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] ClusterAdminRpcService - Failed to add replica for shard ShardName{value=no-leader}, type Operational org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1010) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleReceive(ShardManager.java:243) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:272) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:233) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:245) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 08:41:09,499 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-587561008] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:09,501 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Shard created, persistent : true 08:41:09,502 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#-519410704 created and ready for shard:member-2-shard-cars-oper_testAddReplicasForAllShards 08:41:09,502 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#-519410704] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:09,503 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 1 08:41:09,503 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:41:09,505 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#1104158069 for shard member-2-shard-cars-config_testAddReplicasForAllShards 08:41:09,505 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#1034732319] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:09,505 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Shard created, persistent : true 08:41:09,506 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#18213736 created and ready for shard:member-2-shard-cars-config_testAddReplicasForAllShards 08:41:09,506 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#18213736] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:09,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-668536975 for shard member-2-shard-cars-oper_testAddReplicasForAllShards 08:41:09,509 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1630194518 for shard member-2-shard-people-config_testAddReplicasForAllShards 08:41:09,509 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Shard created, persistent : true 08:41:09,509 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#766460547 for shard member-2-shard-pets-config_testAddReplicasForAllShards 08:41:09,510 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-notifier#1489533344 created and ready for shard:member-2-shard-people-config_testAddReplicasForAllShards 08:41:09,511 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-people-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:41:09,512 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-people-config_testAddReplicasForAllShards: journal open: applyTo=0 08:41:09,513 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#522097454 for shard member-2-shard-people-oper_testAddReplicasForAllShards 08:41:09,514 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Shard created, persistent : true 08:41:09,515 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:09,515 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActor - member-2-shard-people-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:41:09,516 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-notifier#284190662 created and ready for shard:member-2-shard-people-oper_testAddReplicasForAllShards 08:41:09,517 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-people-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:41:09,518 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:41:09,518 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testAddReplicasForAllShards is initialized 08:41:09,518 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:09,518 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:09,518 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 08:41:09,519 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 08:41:09,519 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testAddReplicasForAllShards from null to Follower 08:41:09,520 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddReplicasForAllShards: journal open: applyTo=0 08:41:09,526 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-cars-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:41:09,526 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:09,527 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddReplicasForAllShards: journal open: applyTo=0 08:41:09,524 AM [cluster-test-shard-dispatcher-18] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-config_testAddReplicasForAllShards 08:41:09,521 AM [cluster-test-shard-dispatcher-17] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddReplicasForAllShards 08:41:09,523 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testAddReplicasForAllShards: journal open: applyTo=0 08:41:09,528 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:09,530 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:41:09,530 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:41:09,530 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddReplicasForAllShards is initialized 08:41:09,530 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:09,530 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:09,530 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 08:41:09,530 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 08:41:09,523 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddReplicasForAllShards 08:41:09,531 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddReplicasForAllShards from null to Follower 08:41:09,531 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:41:09,532 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:41:09,532 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddReplicasForAllShards is initialized 08:41:09,532 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:09,532 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:09,532 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 08:41:09,532 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 08:41:09,532 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddReplicasForAllShards from null to Follower 08:41:09,530 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:09,533 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-oper_testAddReplicasForAllShards 08:41:09,533 AM [cluster-test-shard-dispatcher-15] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-pets-config_testAddReplicasForAllShards 08:41:09,535 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:41:09,535 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:41:09,535 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:09,535 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:41:09,535 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testAddReplicasForAllShards is initialized 08:41:09,536 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 08:41:09,536 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 08:41:09,536 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testAddReplicasForAllShards from null to Follower 08:41:09,545 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:41:09,543 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:41:09,546 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:09,544 AM [cluster-test-shard-dispatcher-18] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:41:09,546 AM [cluster-test-shard-dispatcher-18] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:09,543 AM [cluster-test-shard-dispatcher-15] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:41:09,546 AM [cluster-test-shard-dispatcher-15] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:09,546 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:09,548 AM [cluster-test-shard-dispatcher-17] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:41:09,548 AM [cluster-test-shard-dispatcher-17] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:09,562 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:41:09,562 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:41:09,563 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:41:09,569 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-30T08:41:09.546801404Z 08:41:09,562 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:41:09,562 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:41:09,573 AM [cluster-test-shard-dispatcher-17] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-30T08:41:09.548303926Z 08:41:09,575 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-people-config_testAddReplicasForAllShards 08:41:09,575 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddReplicasForAllShards 08:41:09,575 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-people-oper_testAddReplicasForAllShards 08:41:09,575 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddReplicasForAllShards 08:41:09,575 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-pets-config_testAddReplicasForAllShards 08:41:09,577 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:09,577 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:09,578 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: snapshot is durable as of 2025-08-30T08:41:09.546375511Z 08:41:09,579 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2a6f3abe 08:41:09,579 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 08:41:09,578 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:09,579 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4f98e75b 08:41:09,579 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 08:41:09,579 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7bb717f 08:41:09,579 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 08:41:09,580 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-30T08:41:09.546376692Z 08:41:09,580 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:09,580 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 08:41:09,580 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:09,581 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@344f3776 08:41:09,581 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 08:41:09,581 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@16cef5d6 08:41:09,581 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:41:09,582 AM [cluster-test-shard-dispatcher-32] [INFO] SnapshotManager - member-2-shard-people-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-30T08:41:09.579471678Z 08:41:09,583 AM [cluster-test-shard-dispatcher-32] [INFO] ShardSnapshotCohort - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot 08:41:09,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-people-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:09,583 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: snapshot is durable as of 2025-08-30T08:41:09.546970858Z 08:41:09,584 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddReplicasForAllShards: snapshot is durable as of 2025-08-30T08:41:09.579032316Z 08:41:09,584 AM [cluster-test-shard-dispatcher-30] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot 08:41:09,584 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:09,585 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-2-shard-pets-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-30T08:41:09.580585790Z 08:41:09,585 AM [cluster-test-shard-dispatcher-29] [INFO] ShardSnapshotCohort - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot 08:41:09,585 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-pets-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:09,590 AM [cluster-test-shard-dispatcher-24] [INFO] SnapshotManager - member-2-shard-people-oper_testAddReplicasForAllShards: snapshot is durable as of 2025-08-30T08:41:09.579545730Z 08:41:09,590 AM [cluster-test-shard-dispatcher-24] [INFO] ShardSnapshotCohort - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot 08:41:09,590 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-2-shard-people-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:09,594 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-2-shard-cars-config_testAddReplicasForAllShards: snapshot is durable as of 2025-08-30T08:41:09.581029263Z 08:41:09,595 AM [cluster-test-shard-dispatcher-31] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot 08:41:09,595 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:09,611 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-pets-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#419271879] with name member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher 08:41:09,611 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#-1305861561] with name member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher 08:41:09,612 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-pets-config_testAddReplicasForAllShards: state snapshot applied in 27.05 ms 08:41:09,612 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddReplicasForAllShards: state snapshot applied in 17.35 ms 08:41:09,612 AM [cluster-test-shard-dispatcher-31] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddReplicasForAllShards: Done applying snapshot 08:41:09,612 AM [cluster-test-shard-dispatcher-29] [INFO] ShardSnapshotCohort - member-2-shard-pets-config_testAddReplicasForAllShards: Done applying snapshot 08:41:09,614 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#-1305861561] 08:41:09,614 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#419271879] 08:41:09,615 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#1911435631] with name member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher 08:41:09,615 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddReplicasForAllShards: state snapshot applied in 30.51 ms 08:41:09,615 AM [cluster-test-shard-dispatcher-30] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddReplicasForAllShards: Done applying snapshot 08:41:09,615 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddReplicasForAllShards: publishChanges: rootPath: / 08:41:09,615 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-pets-config_testAddReplicasForAllShards: publishChanges: rootPath: / 08:41:09,615 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#1554290112] with name member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher 08:41:09,615 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-2-shard-people-oper_testAddReplicasForAllShards: state snapshot applied in 24.99 ms 08:41:09,615 AM [cluster-test-shard-dispatcher-24] [INFO] ShardSnapshotCohort - member-2-shard-people-oper_testAddReplicasForAllShards: Done applying snapshot 08:41:09,615 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#1554290112] 08:41:09,618 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#1911435631] 08:41:09,618 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 08:41:09,618 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#1208254231] 08:41:09,621 AM [cluster-test-shard-dispatcher-29] [INFO] AbstractLeader - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:41:09,622 AM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:41:09,622 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher: 6.947 ms 08:41:09,623 AM [cluster-test-shard-dispatcher-17] [INFO] AbstractLeader - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-pets-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:41:09,623 AM [cluster-test-shard-dispatcher-15] [INFO] AbstractLeader - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:41:09,623 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher: 5.699 ms 08:41:09,624 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-pets-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher: 8.674 ms 08:41:09,624 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 08:41:09,624 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#1208254231] with name member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher 08:41:09,624 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-config_testAddReplicasForAllShards: state snapshot applied in 41.71 ms 08:41:09,625 AM [cluster-test-shard-dispatcher-32] [INFO] ShardSnapshotCohort - member-2-shard-people-config_testAddReplicasForAllShards: Done applying snapshot 08:41:09,625 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher: 184.3 μs 08:41:09,626 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-config_testAddReplicasForAllShards: publishChanges: rootPath: / 08:41:09,626 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher: 97.99 μs 08:41:09,649 AM [cluster-test-shard-dispatcher-17] [INFO] AbstractLeader - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:41:09,660 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@380847a3 08:41:09,660 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-oper_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#522097454 08:41:09,660 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard people 08:41:09,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-pets-config_testAddReplicasForAllShards} for shard pets from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#766460547 08:41:09,661 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard pets 08:41:09,661 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-pets-config_testAddReplicasForAllShards 08:41:09,661 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-oper_testAddReplicasForAllShards 08:41:09,661 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets] 08:41:09,661 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [people] 08:41:09,664 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 08:41:09,664 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:09,666 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:09,667 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 08:41:09,667 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:09,667 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-668536975 08:41:09,668 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 08:41:09,668 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddReplicasForAllShards 08:41:09,668 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars, people] 08:41:09,668 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 08:41:09,668 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:09,670 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:41:09,670 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 2 08:41:09,670 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@55390c68 08:41:09,670 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done true 08:41:09,671 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done true 08:41:09,671 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:41:09,671 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 3 08:41:09,672 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=pets}, type Config 08:41:09,672 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2b5042a1 08:41:09,672 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 4 08:41:09,672 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:41:09,672 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-config_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#-1630194518 08:41:09,672 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard people 08:41:09,672 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-config_testAddReplicasForAllShards 08:41:09,672 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, people] 08:41:09,672 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 08:41:09,673 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:09,673 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 08:41:09,676 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:09,676 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-pets-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:09,677 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:09,679 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#1104158069 08:41:09,679 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 08:41:09,679 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddReplicasForAllShards 08:41:09,680 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars, people] 08:41:09,680 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 08:41:09,680 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:09,680 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:09,681 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:41:09,681 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 5 08:41:09,682 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:41:09,682 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5dae29d8 08:41:09,682 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 6 08:41:09,682 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:41:09,682 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1aa47d76 08:41:09,682 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:41:09,682 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done true 08:41:09,682 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done true 08:41:09,682 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done true 08:41:09,683 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 08:41:09,684 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 08:41:09,684 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 08:41:09,684 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 08:41:09,693 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1173380323] 08:41:09,694 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:09,695 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#1034732319] 08:41:09,697 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#1034732319] 08:41:09,697 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1173380323] 08:41:09,697 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:41:09,698 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#-978126105] 08:41:09,698 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#-978126105] 08:41:09,698 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1173380323] 08:41:09,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 08:41:09,698 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-459594008] 08:41:09,698 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-459594008] 08:41:09,698 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#395020930] 08:41:09,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:41:09,699 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-587561008] 08:41:09,699 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-587561008] 08:41:09,699 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#395020930] 08:41:09,699 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 08:41:09,699 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#-1110226382] 08:41:09,699 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#-1110226382] 08:41:09,700 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 08:41:09,700 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testAddReplicasForAllShards 08:41:09,700 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddReplicasForAllShards 08:41:09,700 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testAddReplicasForAllShards 08:41:09,700 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 08:41:09,701 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Stopping Shard 08:41:09,701 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Stopping Shard 08:41:09,702 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testAddReplicasForAllShards: journal closed 08:41:09,702 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Stopping Shard 08:41:09,702 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddReplicasForAllShards: journal closed 08:41:09,702 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-people-config_testAddReplicasForAllShards: journal closed 08:41:09,703 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:41:09,703 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:41:09,711 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 08:41:09,711 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddReplicasForAllShards 08:41:09,711 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testAddReplicasForAllShards 08:41:09,712 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 08:41:09,712 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 3 shards 08:41:09,712 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Stopping Shard 08:41:09,713 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Stopping Shard 08:41:09,713 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Stopping Shard 08:41:09,713 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testAddReplicasForAllShards: journal closed 08:41:09,713 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-no-leader-oper_testAddReplicasForAllShards: journal closed 08:41:09,713 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddReplicasForAllShards: journal closed 08:41:09,713 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:41:09,713 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:41:09,715 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:09,716 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:09,740 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:09,740 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:09,742 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:41:09,760 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 08:41:09,761 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testAddReplicasForAllShards 08:41:09,761 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddReplicasForAllShards 08:41:09,761 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testAddReplicasForAllShards 08:41:09,761 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 08:41:09,765 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6b16c80e 08:41:09,765 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@38a00851 08:41:09,765 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@46bfcac5 08:41:09,766 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4b48d761 08:41:09,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1a4b7693 08:41:09,766 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4b48d761 08:41:09,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1a4b7693 08:41:09,766 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 08:41:09,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 08:41:09,767 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@76fb59cf 08:41:09,767 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-pets-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@76fb59cf 08:41:09,767 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 08:41:10,745 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-5511948437593515863] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:41:14,9 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:14,327 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:14,457 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:14,961 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:15,284 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:15,459 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 08:41:15,461 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:15,461 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:15,472 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:15,924 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:16,247 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:16,426 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:16,492 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:16,887 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:17,206 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:17,516 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:17,521 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:17,844 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:18,166 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:18,434 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:18,534 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:18,793 AM [cluster-test-shard-dispatcher-17] [WARN] Leader - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 08:41:18,794 AM [cluster-test-shard-dispatcher-15] [WARN] Leader - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 08:41:18,794 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer aborted 08:41:18,794 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer aborted 08:41:18,794 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.030 s 08:41:18,794 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.030 s 08:41:18,794 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In unpauseLeader 08:41:18,794 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In unpauseLeader 08:41:18,795 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:41:18,795 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:41:18,795 AM [cluster-test-shard-dispatcher-29] [WARN] Leader - member-1-shard-people-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 08:41:18,795 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 08:41:18,795 AM [cluster-test-shard-dispatcher-11] [WARN] Leader - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 08:41:18,795 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 08:41:18,795 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:41:18,795 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:41:18,795 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:41:18,796 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 08:41:18,796 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 08:41:18,796 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:41:18,796 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:41:18,796 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 08:41:18,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:41:18,796 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Stopping Shard 08:41:18,796 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 08:41:18,797 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Stopping Shard 08:41:18,797 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddReplicasForAllShards: journal closed 08:41:18,798 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-config_testAddReplicasForAllShards: journal closed 08:41:18,813 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer aborted 08:41:18,813 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.049 s 08:41:18,813 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In unpauseLeader 08:41:18,814 AM [cluster-test-shard-dispatcher-11] [WARN] Leader - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-pets-config_testAddReplicasForAllShards from Leader to IsolatedLeader 08:41:18,814 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:41:18,814 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:41:18,815 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 08:41:18,815 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Leader to IsolatedLeader 08:41:18,815 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Stopping Shard 08:41:18,816 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testAddReplicasForAllShards: journal closed 08:41:18,817 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:41:18,817 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:41:18,817 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 08:41:18,819 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddReplicasForAllShards 08:41:18,819 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testAddReplicasForAllShards 08:41:18,819 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:41:18,820 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Stopping Shard 08:41:18,820 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Stopping Shard 08:41:18,820 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testAddReplicasForAllShards: journal closed 08:41:18,821 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddReplicasForAllShards: journal closed 08:41:18,821 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:41:18,821 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:41:18,823 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:18,823 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:18,826 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:41:18.829] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:19,13 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:41:19,33 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [4955549542902598102] 08:41:19,34 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:41:19,38 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:41:19,38 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:41:19,39 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:19,40 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:19,60 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:19,60 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:41:19,60 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:41:19,60 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:41:19,63 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:19,66 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 08:41:19,71 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1551717437] 08:41:19,72 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1551717437] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,73 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:41:19,73 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:41:19,73 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:41:19,73 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStates 08:41:19,73 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStates 08:41:19,74 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:19,74 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:41:19,74 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:41:19,77 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:41:19,78 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@63d0caae} 08:41:19,79 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 08:41:19,79 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:19,79 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStates 08:41:19,79 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:19,81 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:41:19,81 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:41:19,81 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates on shard member-1-shard-cars-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:19,81 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStates 08:41:19,81 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:19,83 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 08:41:19,84 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,85 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 08:41:19,85 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:41:19,85 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:41:19,85 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates on shard member-1-shard-people-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-64649115] 08:41:19,86 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-64649115] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,86 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 08:41:19,87 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#469586531] 08:41:19,87 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#469586531] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,88 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:41:19,88 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:41:19,88 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:41:19,88 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:19,89 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStates 08:41:19,93 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:41:19,95 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:41:19,97 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:19,97 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:19,97 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:41:19,97 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 08:41:19,98 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#-2136268823 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStates 08:41:19,98 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#-2136268823] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,99 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 08:41:19,103 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:19,103 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:19,105 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#682653072 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStates 08:41:19,105 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#682653072] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,107 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:41:19,107 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStates is initialized 08:41:19,107 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:41:19,107 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStates is initialized 08:41:19,107 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:19,107 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:19,107 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:19,107 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:19,108 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 08:41:19,108 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 08:41:19,108 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from null to Follower 08:41:19,108 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 08:41:19,108 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:41:19,108 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 08:41:19,109 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from null to Follower 08:41:19,109 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@5119ebad} 08:41:19,109 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 08:41:19,110 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:19,110 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:19,111 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:19,111 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:19,111 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-1-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1342089477] 08:41:19,111 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStates 08:41:19,111 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 08:41:19,111 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1342089477] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,112 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#-132380446 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:19,112 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#-132380446] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,112 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:41:19,112 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:41:19,112 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-1-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1471809977] 08:41:19,113 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 08:41:19,113 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1471809977] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,113 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#1824717073 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStates 08:41:19,113 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:41:19,113 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#1824717073] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,114 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 08:41:19,115 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:19,116 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:19,116 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:19,116 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 08:41:19,116 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:41:19,116 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStates is initialized 08:41:19,116 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 08:41:19,116 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:41:19,116 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from null to Follower 08:41:19,117 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 08:41:19,118 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:19,119 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:41:19,119 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStates is initialized 08:41:19,119 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:19,119 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:19,119 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 08:41:19,119 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 08:41:19,119 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from null to Follower 08:41:19,124 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:19,139 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-6406873741924881145] 08:41:19,139 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.1.5] ... 08:41:19,142 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:19,142 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:41:19,146 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:19,146 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:19,164 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:19,167 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:19,170 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 08:41:19,174 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1918832280] 08:41:19,176 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:41:19,176 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:41:19,176 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:41:19,176 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStates 08:41:19,176 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1918832280] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,176 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStates 08:41:19,191 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:41:19,192 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@36da8c4a} 08:41:19,193 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 08:41:19,193 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:19,193 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStates 08:41:19,193 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:19,197 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:19,200 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 08:41:19,202 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStates 08:41:19,202 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 08:41:19,203 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1629026694] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,203 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:41:19,203 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#1736641949 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStates 08:41:19,203 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#1736641949] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,204 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 08:41:19,205 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 08:41:19,205 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#495692478] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,206 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:41:19,206 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#-973578689 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStates 08:41:19,206 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#-973578689] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,206 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 08:41:19,211 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-432985467] 08:41:19,211 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-432985467] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,211 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:41:19,212 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:41:19,212 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:41:19,212 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStates 08:41:19,212 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStates 08:41:19,213 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:19,214 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:41:19,214 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStates is initialized 08:41:19,214 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:19,214 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:19,215 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 08:41:19,215 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 08:41:19,215 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from null to Follower 08:41:19,218 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:19,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:41:19,219 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:19,219 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:19,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStates is initialized 08:41:19,219 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 08:41:19,219 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 08:41:19,219 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from null to Follower 08:41:19,220 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:41:19,221 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@166a09c3} 08:41:19,221 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 08:41:19,226 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:19,226 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStates 08:41:19,228 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStates 08:41:19,229 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 08:41:19,230 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 08:41:19,230 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1807983399] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,230 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#905409669] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,231 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#1831006141 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStates 08:41:19,231 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#970615442 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStates 08:41:19,231 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:41:19,231 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:41:19,231 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#1831006141] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,231 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#970615442] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,232 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 08:41:19,233 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 08:41:19,238 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:19,238 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:19,239 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:19,239 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:19,239 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:19,239 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:19,239 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 08:41:19,240 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 08:41:19,240 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:41:19,242 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:41:19,242 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStates is initialized 08:41:19,242 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:41:19,243 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStates is initialized 08:41:19,243 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 08:41:19,243 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from null to Follower 08:41:19,243 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 08:41:19,243 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from null to Follower 08:41:19,245 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:41:19,247 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:19,247 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:41:19,247 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:41:19,247 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:41:19,247 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:41:19,247 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:41:19,247 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:41:19,247 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:19,248 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:19,248 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:41:19,248 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:41:19,248 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:41:19,248 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:41:19,248 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:19,259 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:19,279 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-736689609200176519] 08:41:19,281 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.1.5] ... 08:41:19,284 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:19,285 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:41:19,286 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:19,286 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:19,305 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:19,316 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:19,319 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 08:41:19,338 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1757839869] 08:41:19,344 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1757839869] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,344 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:41:19,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:41:19,344 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:41:19,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStates 08:41:19,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStates 08:41:19,351 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:41:19,357 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:41:19,358 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:19,358 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:41:19,358 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:41:19,364 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:41:19,365 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@76a8d47b} 08:41:19,365 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 08:41:19,366 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:19,366 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStates 08:41:19,366 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:19,367 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:41:19,367 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:41:19,368 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates on shard member-3-shard-cars-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#772274063] 08:41:19,368 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStates 08:41:19,369 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:19,371 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 08:41:19,373 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 08:41:19,372 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#772274063] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,375 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#-406498616 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStates 08:41:19,375 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#-406498616] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,376 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:41:19,377 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 08:41:19,377 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:41:19,377 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:41:19,377 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates on shard member-3-shard-people-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#457043930] 08:41:19,377 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 08:41:19,377 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#457043930] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,378 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:41:19,379 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 08:41:19,379 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#-2080650429 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStates 08:41:19,379 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#-2080650429] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,380 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1800702177] 08:41:19,381 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:19,381 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:41:19,381 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1800702177] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,381 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:41:19,381 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:41:19,381 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:19,381 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:41:19,382 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:41:19,382 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStates is initialized 08:41:19,382 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:19,382 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:19,382 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 08:41:19,382 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:41:19,382 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 08:41:19,382 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from null to Follower 08:41:19,384 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 08:41:19,385 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:19,385 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:41:19,385 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStates 08:41:19,385 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:41:19,385 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStates is initialized 08:41:19,385 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:19,385 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:19,386 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:19,386 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:41:19,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@688616a1} 08:41:19,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:19,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:19,386 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:19,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:19,387 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-3-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1437114988] 08:41:19,387 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStates 08:41:19,387 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 08:41:19,387 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1437114988] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,385 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:19,388 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 08:41:19,388 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#-739152054 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:19,388 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#-739152054] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,388 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:41:19,388 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 08:41:19,388 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from null to Follower 08:41:19,388 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:41:19,388 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:41:19,388 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:41:19,388 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-3-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1096776169] 08:41:19,389 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1096776169] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,389 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 08:41:19,389 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 08:41:19,390 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:41:19,390 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#1315194790 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStates 08:41:19,390 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#1315194790] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:19,391 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 08:41:19,392 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:19,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:41:19,392 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStates is initialized 08:41:19,392 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:19,393 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:19,393 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 08:41:19,393 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:19,393 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:19,393 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 08:41:19,393 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from null to Follower 08:41:19,393 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:41:19,393 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStates is initialized 08:41:19,394 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:19,394 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:19,394 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 08:41:19,394 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:41:19,394 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 08:41:19,394 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from null to Follower 08:41:20,60 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:41:20,60 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:41:20,61 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:20,62 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:20,62 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:20,62 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:41:20,63 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:41:20,63 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:41:20,62 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,63 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:20,63 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:41:20,63 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:41:20,63 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:41:20,63 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:41:20,63 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:41:20,63 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:41:20,63 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:41:20,75 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:20,76 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:20,76 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:20,76 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:20,76 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:41:20,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:41:20,78 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,162 AM [cluster-test-shard-dispatcher-31] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 08:41:20,163 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:41:20,163 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,163 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,163 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,163 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 08:41:20,169 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:41:20,183 AM [cluster-test-shard-dispatcher-31] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:41:20,185 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:41:20,185 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,186 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,186 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 08:41:20,186 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@76a33ce 08:41:20,186 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to PreLeader 08:41:20,186 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:20,186 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:20,186 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:20,187 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:41:20,186 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:20,187 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:41:20,187 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:41:20,191 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 08:41:20,192 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 08:41:20,192 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:41:20,192 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,192 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,193 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,193 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 08:41:20,193 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 08:41:20,193 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 08:41:20,193 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@757cad00 08:41:20,194 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done false 08:41:20,194 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done true 08:41:20,194 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@20b52104 08:41:20,195 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:41:20,203 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:41:20,203 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:41:20,203 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,204 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,204 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 08:41:20,204 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6c8ba602 08:41:20,204 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to PreLeader 08:41:20,206 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 08:41:20,206 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:41:20,206 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,206 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 08:41:20,206 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from PreLeader to Leader 08:41:20,207 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2dd3e1f6 08:41:20,207 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 08:41:20,207 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 08:41:20,209 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6641f5c1 08:41:20,209 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 08:41:20,209 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 08:41:20,210 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 08:41:20,210 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:41:20,210 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,210 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,210 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,210 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 08:41:20,217 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:41:20,219 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:41:20,219 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,219 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 08:41:20,219 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from PreLeader to Leader 08:41:20,221 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 08:41:20,221 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:41:20,221 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,221 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,221 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,221 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 08:41:20,223 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:41:20,225 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:41:20,225 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:41:20,225 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,226 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,226 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 08:41:20,226 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7156cf03 08:41:20,226 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to PreLeader 08:41:20,228 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 08:41:20,229 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6ceeebcd 08:41:20,229 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:41:20,229 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 08:41:20,229 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 08:41:20,231 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done false 08:41:20,231 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done true 08:41:20,231 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3f19f022 08:41:20,232 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:41:20,232 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:41:20,232 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:41:20,232 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,232 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,232 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 08:41:20,232 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4f778f00 08:41:20,233 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to PreLeader 08:41:20,235 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 08:41:20,238 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7f97d6aa 08:41:20,238 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:41:20,238 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done false 08:41:20,238 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done true 08:41:20,239 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6fa6068f 08:41:20,239 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:41:20,239 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 08:41:20,240 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 08:41:20,242 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:41:20,243 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,243 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 08:41:20,243 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from PreLeader to Leader 08:41:20,243 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:41:20,249 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:41:20,249 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,249 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 08:41:20,249 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from PreLeader to Leader 08:41:20,249 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:41:20,249 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 08:41:20,249 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:41:20,249 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 08:41:20,249 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:41:20,249 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 08:41:20,249 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:41:20,249 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 08:41:20,250 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1551717437] 08:41:20,250 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:20,250 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,250 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,253 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 08:41:20,254 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 08:41:20,254 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 08:41:20,254 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:41:20,254 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:41:20,256 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:41:20,256 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:41:20,257 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:41:20,257 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:20,257 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:41:20,257 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:41:20,261 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,261 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,261 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,264 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,289 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:41:20,290 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@27c63f1f 08:41:20,290 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@ce25ae3 08:41:20,290 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@ce25ae3 08:41:20,290 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:20,290 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,290 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 08:41:20,290 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:41:20,290 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 08:41:20,290 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,291 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 08:41:20,291 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:41:20,291 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 08:41:20,291 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,291 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 08:41:20,291 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:41:20,291 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 08:41:20,292 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,292 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 08:41:20,292 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:41:20,292 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 08:41:20,292 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@26b10748 08:41:20,293 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1551717437] 08:41:20,294 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:20,294 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@195748a8 08:41:20,294 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,294 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,294 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1551717437] 08:41:20,295 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:41:20,295 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-64649115] 08:41:20,295 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-64649115] 08:41:20,295 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#469586531] 08:41:20,295 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:41:20,296 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1342089477] 08:41:20,296 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1342089477] 08:41:20,296 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#469586531] 08:41:20,296 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:41:20,296 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:41:20,296 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1471809977] 08:41:20,296 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7c94e2b 08:41:20,297 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1471809977] 08:41:20,297 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:41:20,297 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1918832280] 08:41:20,297 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:41:20,297 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:20,297 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@40297f8b 08:41:20,297 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@40297f8b 08:41:20,297 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1629026694] 08:41:20,297 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1629026694] 08:41:20,297 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@404dedd4 08:41:20,298 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@20e12019 08:41:20,298 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@404dedd4 08:41:20,298 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:20,298 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3f968296 08:41:20,298 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1918832280] 08:41:20,298 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:41:20,297 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3eb7ae4f 08:41:20,298 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#495692478] 08:41:20,298 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3eb7ae4f 08:41:20,298 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:20,298 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#495692478] 08:41:20,297 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:20,298 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-432985467] 08:41:20,298 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:41:20,298 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1807983399] 08:41:20,298 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1807983399] 08:41:20,298 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-432985467] 08:41:20,299 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 08:41:20,299 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#905409669] 08:41:20,299 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#905409669] 08:41:20,299 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1757839869] 08:41:20,299 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:20,299 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#772274063] 08:41:20,299 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#772274063] 08:41:20,299 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1757839869] 08:41:20,300 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:41:20,300 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#457043930] 08:41:20,300 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:41:20,300 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#457043930] 08:41:20,300 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1800702177] 08:41:20,300 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:41:20,300 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1437114988] 08:41:20,300 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1437114988] 08:41:20,300 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1800702177] 08:41:20,301 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:41:20,301 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1096776169] 08:41:20,302 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5a4801b8 08:41:20,302 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@9cfb7ed 08:41:20,302 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@aaf447e 08:41:20,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1096776169] 08:41:20,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1551717437] 08:41:20,304 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:20,304 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,306 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:41:20,309 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@60464fa 08:41:20,310 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3062fca3 08:41:20,311 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5175f90d 08:41:20,311 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:41:20,315 AM [cluster-test-shard-dispatcher-30] [INFO] Candidate - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 08:41:20,315 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 08:41:20,315 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,315 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,315 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,315 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1d61954a 08:41:20,316 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 08:41:20,316 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:20,316 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,316 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,316 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 08:41:20,316 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@c6865d5 08:41:20,316 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 08:41:20,317 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:41:20,317 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:41:20,318 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 08:41:20,318 AM [cluster-test-shard-dispatcher-29] [INFO] Candidate - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 08:41:20,318 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 08:41:20,318 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,318 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,318 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,318 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@26f4b2a4 08:41:20,318 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 08:41:20,318 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:20,318 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,319 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,319 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 08:41:20,319 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@59b68001 08:41:20,319 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 08:41:20,319 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:41:20,320 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 08:41:20,320 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 08:41:20,320 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2dde41fd 08:41:20,322 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 08:41:20,322 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@240a9924 08:41:20,322 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:41:20,324 AM [cluster-test-shard-dispatcher-31] [INFO] AbstractLeader - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 08:41:20,324 AM [cluster-test-shard-dispatcher-15] [INFO] AbstractLeader - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 08:41:20,328 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 08:41:20,328 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 08:41:20,330 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 08:41:20,331 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,331 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,331 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 08:41:20,331 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@79b842d4 08:41:20,331 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 08:41:20,331 AM [cluster-test-shard-dispatcher-29] [INFO] Candidate - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 08:41:20,331 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 08:41:20,331 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 08:41:20,331 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 08:41:20,331 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,331 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,331 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,331 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,331 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 08:41:20,331 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,331 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,331 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,331 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,331 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7244f983 08:41:20,331 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 08:41:20,331 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 08:41:20,331 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@25406988 08:41:20,331 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@57c27085 08:41:20,332 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 08:41:20,332 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 08:41:20,332 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:20,332 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,332 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:20,332 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,332 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,332 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 08:41:20,332 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,332 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5e5b222f 08:41:20,332 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 08:41:20,332 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 08:41:20,332 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@77d50e25 08:41:20,332 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 08:41:20,332 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:41:20,333 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 08:41:20,334 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 08:41:20,335 AM [cluster-test-shard-dispatcher-15] [INFO] AbstractLeader - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 08:41:20,336 AM [cluster-test-shard-dispatcher-10] [INFO] AbstractLeader - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 08:41:20,337 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3ed509dc 08:41:20,337 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 08:41:20,338 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4a51a134 08:41:20,338 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:41:20,339 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 08:41:20,339 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 08:41:20,339 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,339 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,339 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 08:41:20,339 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 08:41:20,339 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,339 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,339 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 08:41:20,340 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ea488de 08:41:20,340 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Leader to Follower 08:41:20,340 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@86cf158 08:41:20,340 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 08:41:20,341 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 08:41:20,341 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 08:41:20,356 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,407 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,458 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,508 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,559 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,610 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,641 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader changed to member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,642 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-oper_testFlipMemberVotingStates in 351.8 ms 08:41:20,642 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7791d1d 08:41:20,642 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 08:41:20,642 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,642 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-oper_testFlipMemberVotingStates in 345.9 ms 08:41:20,643 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 08:41:20,643 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2ae0f18 08:41:20,643 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:41:20,651 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader changed to member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,651 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-config_testFlipMemberVotingStates in 353.9 ms 08:41:20,651 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader changed to member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,651 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 08:41:20,651 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-config_testFlipMemberVotingStates in 354.2 ms 08:41:20,652 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4a8ca3cf 08:41:20,652 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 08:41:20,652 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 08:41:20,652 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 08:41:20,652 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2c361f10 08:41:20,652 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:41:20,657 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 08:41:20,657 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 08:41:20,660 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,661 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#469586531] 08:41:20,661 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:41:20,661 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1342089477] 08:41:20,661 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1342089477] 08:41:20,661 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 08:41:20,661 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 08:41:20,662 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:41:20,662 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 08:41:20,662 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:41:20,662 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:41:20,662 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:20,662 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:41:20,662 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:41:20,663 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:41:20,663 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:41:20,663 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,663 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,664 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,664 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 08:41:20,664 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:41:20,664 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,664 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 08:41:20,664 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,664 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 08:41:20,665 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:41:20,665 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 08:41:20,665 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,665 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,665 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 08:41:20,665 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:41:20,665 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 08:41:20,666 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,666 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 08:41:20,666 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:41:20,666 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 08:41:20,667 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1551717437] 08:41:20,667 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:20,667 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,667 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,673 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-3-shard-cars-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:41:20,673 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorVotingConfigSupport - member-3-shard-people-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:41:20,673 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3cf378e8 08:41:20,673 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3cf378e8 08:41:20,673 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:20,673 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorVotingConfigSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:41:20,673 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@70e2193a 08:41:20,673 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@70e2193a 08:41:20,673 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:20,673 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6308ed75 08:41:20,673 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@121f4fc4 08:41:20,673 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@121f4fc4 08:41:20,673 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:20,674 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3f565070 08:41:20,675 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:41:20,676 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:41:20,677 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:41:20,677 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorVotingConfigSupport - member-3-shard-people-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:41:20,677 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@d2e95d3 08:41:20,677 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@d2e95d3 08:41:20,677 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:20,677 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:41:20,678 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@59ccd6f6 08:41:20,691 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2214dc09 08:41:20,700 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@465a22a3 08:41:20,700 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@41524832 08:41:20,701 AM [cluster-test-shard-dispatcher-25] [INFO] Candidate - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 08:41:20,701 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorBehavior - member-2-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 08:41:20,701 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,701 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,701 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5faa3879 08:41:20,701 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,701 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@37e0008e 08:41:20,701 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 08:41:20,701 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 08:41:20,701 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 08:41:20,701 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,701 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,701 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,701 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@49dca91c 08:41:20,702 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 08:41:20,702 AM [cluster-test-shard-dispatcher-30] [INFO] Candidate - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 08:41:20,702 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 08:41:20,702 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,702 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,702 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,702 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6cf98bfb 08:41:20,702 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 08:41:20,702 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 08:41:20,702 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 08:41:20,702 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,703 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,703 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,703 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e46526c 08:41:20,703 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 08:41:20,705 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5ad66b39 08:41:20,706 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-1-shard-people-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 08:41:20,708 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 08:41:20,708 AM [cluster-test-shard-dispatcher-16] [INFO] Follower - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 08:41:20,708 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 08:41:20,714 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-2-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 08:41:20,714 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,714 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,714 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 08:41:20,715 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6a5bdffd 08:41:20,715 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 08:41:20,716 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 08:41:20,716 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@645be7f7 08:41:20,717 AM [cluster-test-shard-dispatcher-31] [INFO] AbstractLeader - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 08:41:20,718 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 08:41:20,718 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 08:41:20,718 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,718 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,718 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,718 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,718 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 08:41:20,718 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1676dc6d 08:41:20,718 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 08:41:20,718 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 08:41:20,718 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 08:41:20,718 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,718 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:41:20,718 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@21254a5d 08:41:20,718 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,718 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 08:41:20,718 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@d9d832c 08:41:20,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1551717437] 08:41:20,719 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 08:41:20,719 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:41:20,719 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 08:41:20,719 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:41:20,719 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-64649115] 08:41:20,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#-64649115] 08:41:20,720 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#469586531] 08:41:20,720 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:41:20,720 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1342089477] 08:41:20,720 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#-1342089477] 08:41:20,720 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 08:41:20,721 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@43fbad 08:41:20,721 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 08:41:20,721 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:41:20,721 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 08:41:20,721 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 08:41:20,720 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#469586531] 08:41:20,721 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4edcba9a 08:41:20,721 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@edec163 08:41:20,721 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:41:20,721 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:41:20,721 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 08:41:20,721 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1471809977] 08:41:20,721 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1471809977] 08:41:20,721 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,721 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,722 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3321fe1 08:41:20,722 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 08:41:20,722 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1918832280] 08:41:20,722 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 08:41:20,722 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:20,722 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1629026694] 08:41:20,722 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1629026694] 08:41:20,722 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1918832280] 08:41:20,722 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:41:20,723 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#495692478] 08:41:20,723 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#495692478] 08:41:20,724 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-432985467] 08:41:20,724 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:41:20,724 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1807983399] 08:41:20,724 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1807983399] 08:41:20,724 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-432985467] 08:41:20,724 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 08:41:20,725 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#905409669] 08:41:20,725 AM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 08:41:20,725 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#905409669] 08:41:20,725 AM [cluster-test-shard-dispatcher-29] [INFO] AbstractLeader - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 08:41:20,725 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1757839869] 08:41:20,725 AM [cluster-test-shard-dispatcher-31] [INFO] AbstractLeader - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 08:41:20,725 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:20,725 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#772274063] 08:41:20,725 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#772274063] 08:41:20,726 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 08:41:20,726 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 08:41:20,727 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 08:41:20,728 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 08:41:20,728 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,728 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 08:41:20,728 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5129cf70 08:41:20,728 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Leader to Follower 08:41:20,728 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 08:41:20,728 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 08:41:20,728 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,728 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,728 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,728 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,728 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6eedc8c5 08:41:20,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 08:41:20,728 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6a9da585 08:41:20,728 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 08:41:20,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 08:41:20,729 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 08:41:20,729 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1757839869] 08:41:20,729 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:41:20,729 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#457043930] 08:41:20,729 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#457043930] 08:41:20,729 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1800702177] 08:41:20,729 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:41:20,729 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1437114988] 08:41:20,729 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-1437114988] 08:41:20,729 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1800702177] 08:41:20,729 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:41:20,730 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1096776169] 08:41:20,730 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#1096776169] 08:41:20,730 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1551717437] 08:41:20,730 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:20,730 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,730 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#-1915879263] 08:41:20,731 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:41:20,731 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStates 08:41:20,731 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStates 08:41:20,732 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 08:41:20,732 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Stopping Shard 08:41:20,732 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 08:41:20,732 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStates: journal closed 08:41:20,732 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStates: journal closed 08:41:20,733 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:41:20,733 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:41:20,733 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:41:20,734 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStates 08:41:20,734 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStates 08:41:20,734 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:41:20,734 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 08:41:20,734 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 08:41:20,734 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStates: journal closed 08:41:20,734 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStates: journal closed 08:41:20,735 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:41:20,735 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:41:20,736 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:20,737 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:20,745 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:20,745 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:20,745 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:20,746 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:20,748 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:41:20,766 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:41:20,767 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStates 08:41:20,767 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStates 08:41:20,767 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:41:20,768 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78e9dd94 08:41:20,768 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78e9dd94 08:41:20,768 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:20,768 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2d1b6737 08:41:20,768 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2d1b6737 08:41:20,768 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:20,769 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3da255c6 08:41:20,769 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@55c67421 08:41:20,772 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@317456fe 08:41:20,773 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4fe378f5 08:41:20,775 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:41:20,776 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:41:20,781 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 5 08:41:20,782 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 08:41:20,782 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,782 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,782 AM [cluster-test-shard-dispatcher-30] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 5 08:41:20,782 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@583b0490 08:41:20,782 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,782 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 08:41:20,782 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 08:41:20,783 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,783 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,783 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@31dd330d 08:41:20,783 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,783 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 08:41:20,785 AM [cluster-test-shard-dispatcher-25] [INFO] AbstractLeader - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 08:41:20,786 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 08:41:20,792 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorBehavior - member-2-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 08:41:20,792 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,792 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,792 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 08:41:20,793 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 08:41:20,793 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,793 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,793 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@54c9c961 08:41:20,793 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 08:41:20,793 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 08:41:20,793 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@481abe00 08:41:20,793 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Leader to Follower 08:41:20,794 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 08:41:20,795 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,795 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,795 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 08:41:20,795 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,795 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 08:41:20,795 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,795 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 08:41:20,795 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4453669c 08:41:20,795 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 08:41:20,795 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@35eb6ef6 08:41:20,795 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 08:41:20,795 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:41:20,797 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: leader changed to member-1-shard-cars-config_testFlipMemberVotingStates 08:41:20,797 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-config_testFlipMemberVotingStates in 29.70 ms 08:41:20,798 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 08:41:20,798 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@48576d8b 08:41:20,798 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 08:41:20,799 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStates: journal closed 08:41:20,799 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: leader changed to member-1-shard-people-config_testFlipMemberVotingStates 08:41:20,799 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-people-config_testFlipMemberVotingStates in 31.03 ms 08:41:20,799 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 08:41:20,799 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@d2e534d 08:41:20,800 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:41:20,800 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Stopping Shard 08:41:20,800 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStates: journal closed 08:41:20,801 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:41:20,801 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:41:20,801 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:41:20,802 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStates 08:41:20,802 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStates 08:41:20,802 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:41:20,802 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@16b67b69 08:41:20,802 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6d28717 08:41:20,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6d28717 08:41:20,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:20,803 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6737f002 08:41:20,803 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@24e85e3d 08:41:20,803 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@24e85e3d 08:41:20,803 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:20,805 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3bb0a3af 08:41:20,806 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6f9f71 08:41:20,808 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:41:20,808 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:41:20,813 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 5 08:41:20,813 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 08:41:20,814 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,814 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,814 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,814 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@220d406e 08:41:20,814 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 08:41:20,814 AM [cluster-test-shard-dispatcher-30] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 5 08:41:20,814 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 08:41:20,814 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,814 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,814 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:41:20,814 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5f576e66 08:41:20,814 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 08:41:20,816 AM [cluster-test-shard-dispatcher-27] [INFO] AbstractLeader - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 08:41:20,817 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-2-shard-people-oper_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 08:41:20,822 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 08:41:20,822 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,822 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,822 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 08:41:20,822 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@42e980a1 08:41:20,822 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 08:41:20,823 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 08:41:20,823 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:41:20,823 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:20,823 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 08:41:20,823 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7e640c68 08:41:20,823 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 08:41:20,824 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 08:41:20,824 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 08:41:20,824 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,824 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:41:20,824 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,824 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:41:20,824 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 08:41:20,824 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 08:41:20,824 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@31a537a4 08:41:20,824 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 08:41:20,824 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@fb657e4 08:41:20,824 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 08:41:20,824 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:41:20,825 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:20,825 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-oper_testFlipMemberVotingStates in 23.22 ms 08:41:20,826 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: leader changed to member-1-shard-people-oper_testFlipMemberVotingStates 08:41:20,826 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-people-oper_testFlipMemberVotingStates in 23.45 ms 08:41:20,826 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@411ff0b5 08:41:20,826 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 08:41:20,826 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6c262355 08:41:20,826 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:41:20,827 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 08:41:20,827 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 08:41:20,827 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStates: journal closed 08:41:20,828 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 08:41:20,828 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStates: journal closed 08:41:20,828 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:41:20,828 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:41:20,830 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:20,830 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:21,79 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:41:21,758 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-736689609200176519] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:41:21,848 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:21,848 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-6406873741924881145] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:41:21,848 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:21,851 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:41:21.854] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:21,866 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:41:21,867 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStates 08:41:21,867 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStates 08:41:21,867 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:41:21,868 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@10744764 08:41:21,868 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@10744764 08:41:21,868 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:21,869 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3f491c7e 08:41:21,869 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6c09d07d 08:41:21,869 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@767da00a 08:41:21,869 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@767da00a 08:41:21,869 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:22,948 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer aborted 08:41:22,948 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer aborted 08:41:22,948 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.080 s 08:41:22,948 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In unpauseLeader 08:41:22,948 AM [cluster-test-shard-dispatcher-31] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.080 s 08:41:22,948 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In unpauseLeader 08:41:22,950 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Stopping Shard 08:41:22,950 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 08:41:22,950 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStates: journal closed 08:41:22,951 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStates: journal closed 08:41:22,952 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:41:22,952 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:41:22,953 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:41:22,954 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStates 08:41:22,954 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStates 08:41:22,954 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@62ee4475 08:41:22,955 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@62ee4475 08:41:22,955 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:22,955 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@73ded574 08:41:22,955 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@73ded574 08:41:22,955 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:41:22,955 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:41:22,955 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@766225d1 08:41:22,955 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e7f254b 08:41:24,58 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer aborted 08:41:24,58 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer aborted 08:41:24,58 AM [cluster-test-shard-dispatcher-30] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.104 s 08:41:24,58 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In unpauseLeader 08:41:24,58 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.104 s 08:41:24,58 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In unpauseLeader 08:41:24,59 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 08:41:24,59 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 08:41:24,60 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStates: journal closed 08:41:24,60 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStates: journal closed 08:41:24,61 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:41:24,61 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:41:24,63 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:24,63 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:24,67 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:41:24.071] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:24,110 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:41:24,127 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-4242140760639755560] 08:41:24,128 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:41:24,134 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:41:24,134 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:41:24,136 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:24,148 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:24,152 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:24,153 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:41:24,153 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:41:24,153 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:41:24,158 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:24,161 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 08:41:24,169 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#607822915] 08:41:24,170 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#607822915] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,170 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:41:24,172 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:41:24,172 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:41:24,172 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testModuleShardLeaderMovement 08:41:24,172 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testModuleShardLeaderMovement 08:41:24,173 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:24,173 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 08:41:24,173 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 08:41:24,177 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 08:41:24,178 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@63d697a9} 08:41:24,178 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 08:41:24,179 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:24,179 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testModuleShardLeaderMovement 08:41:24,179 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 08:41:24,179 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 08:41:24,179 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement on shard member-1-shard-cars-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320] 08:41:24,180 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testModuleShardLeaderMovement 08:41:24,180 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,180 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 08:41:24,181 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#441900594 created and ready for shard:member-1-shard-cars-config_testModuleShardLeaderMovement 08:41:24,181 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#441900594] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,182 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:41:24,188 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 08:41:24,188 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 08:41:24,188 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement on shard member-1-shard-people-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#1179645952] 08:41:24,188 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Shard created, persistent : true 08:41:24,189 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#1179645952] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,189 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#-468774446 created and ready for shard:member-1-shard-people-config_testModuleShardLeaderMovement 08:41:24,189 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:41:24,189 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#-468774446] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,189 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 08:41:24,190 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testModuleShardLeaderMovement: journal open: applyTo=0 08:41:24,192 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-people-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:24,193 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:24,196 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-people-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:41:24,197 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:41:24,197 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:24,197 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testModuleShardLeaderMovement is initialized 08:41:24,197 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:24,197 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from null to Follower 08:41:24,197 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:41:24,198 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:41:24,198 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testModuleShardLeaderMovement is initialized 08:41:24,198 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:24,198 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:24,198 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 08:41:24,201 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 08:41:24,201 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:24,201 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:24,201 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:24,201 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:41:24,201 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:24,201 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:41:24,201 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: became leader with frontend state for [] 08:41:24,201 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:41:24,202 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 08:41:24,202 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 08:41:24,202 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@18981681 08:41:24,202 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testModuleShardLeaderMovement from Candidate to Leader 08:41:24,202 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:24,202 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:24,202 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:24,202 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:41:24,202 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:24,202 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:41:24,202 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 08:41:24,202 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:41:24,202 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 08:41:24,203 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@412c7c83 08:41:24,203 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 08:41:24,203 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:24,229 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#607822915] 08:41:24,229 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:24,229 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320] 08:41:24,230 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:24,233 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:24,238 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 08:41:24,240 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-2089278840] 08:41:24,241 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-2089278840] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,241 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:41:24,241 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:41:24,241 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:41:24,241 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testModuleShardLeaderMovement 08:41:24,242 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testModuleShardLeaderMovement 08:41:24,242 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:24,242 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 08:41:24,242 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 08:41:24,244 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 08:41:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@4399dd49} 08:41:24,245 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 08:41:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:24,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 08:41:24,246 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 08:41:24,246 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-191031766] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,247 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#523197382 created and ready for shard:member-1-shard-cars-oper_testModuleShardLeaderMovement 08:41:24,247 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:41:24,247 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#523197382] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,248 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 08:41:24,248 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 08:41:24,248 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 08:41:24,249 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement on shard member-1-shard-cars-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-191031766] 08:41:24,249 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testModuleShardLeaderMovement 08:41:24,250 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:24,250 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 08:41:24,250 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 08:41:24,250 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement on shard member-1-shard-people-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#953472559] 08:41:24,251 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#953472559] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,251 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Shard created, persistent : true 08:41:24,252 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#1273600779 created and ready for shard:member-1-shard-people-oper_testModuleShardLeaderMovement 08:41:24,252 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#1273600779] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,252 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:41:24,253 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-people-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:41:24,253 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:24,253 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:24,253 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 08:41:24,253 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:41:24,253 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testModuleShardLeaderMovement is initialized 08:41:24,255 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testModuleShardLeaderMovement: journal open: applyTo=0 08:41:24,255 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 08:41:24,255 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:24,255 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:24,255 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:24,255 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:24,255 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:41:24,255 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 08:41:24,256 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:41:24,256 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:41:24,256 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 08:41:24,256 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@64bf6124 08:41:24,256 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Candidate to Leader 08:41:24,256 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-people-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:24,259 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-people-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:41:24,259 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:41:24,259 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testModuleShardLeaderMovement is initialized 08:41:24,259 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:24,259 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:24,260 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from null to Follower 08:41:24,262 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 08:41:24,262 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:24,262 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:24,262 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:24,262 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:41:24,263 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:24,263 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:41:24,263 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 08:41:24,263 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:41:24,263 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 08:41:24,263 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2c59b299 08:41:24,263 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testModuleShardLeaderMovement from Candidate to Leader 08:41:24,263 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:24,295 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-2089278840] 08:41:24,295 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:41:24,296 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-191031766] 08:41:24,308 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:24,327 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [3369036403065304015] 08:41:24,327 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.1.5] ... 08:41:24,331 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:24,331 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:41:24,339 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:24,339 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:24,353 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:24,355 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:24,357 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 08:41:24,363 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#718051043] 08:41:24,364 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:41:24,364 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:41:24,364 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:41:24,368 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#718051043] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,374 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 08:41:24,376 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@6bc236cd} 08:41:24,376 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 08:41:24,378 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:24,380 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:24,381 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:24,387 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 08:41:24,387 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1102900526] 08:41:24,388 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:41:24,388 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:41:24,388 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:41:24,388 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1102900526] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,392 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 08:41:24,393 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@6505c5c5} 08:41:24,393 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 08:41:24,396 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:24,414 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:24,423 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:41:24,424 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:41:24,432 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:24,432 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:24,432 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:24,432 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:24,438 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [4896329034584688243] 08:41:24,439 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.1.5] ... 08:41:24,441 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:24,441 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:41:24,452 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:24,452 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:24,467 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:24,469 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:24,471 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 08:41:24,474 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1689940554] 08:41:24,475 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1689940554] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,478 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:41:24,478 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:41:24,478 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:41:24,484 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 08:41:24,486 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@7e6a7993} 08:41:24,486 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 08:41:24,486 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:24,486 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:24,488 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:24,490 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 08:41:24,491 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-475863856] 08:41:24,492 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-475863856] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:24,492 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:41:24,492 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:41:24,492 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:41:24,508 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 08:41:24,509 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:41:24,509 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 08:41:24,510 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@70ad882e} 08:41:24,510 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:24,511 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:41:24,522 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:24,522 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:24,522 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:24,522 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:25,157 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:41:25,157 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:41:25,158 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:25,158 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,158 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,158 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement 08:41:25,158 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 08:41:25,158 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:25,158 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:25,158 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,158 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,158 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement 08:41:25,158 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 08:41:25,158 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:25,159 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:25,159 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 08:41:25,159 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 08:41:25,159 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement 08:41:25,159 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 08:41:25,159 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,160 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:25,160 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 08:41:25,160 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 08:41:25,160 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement 08:41:25,160 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 08:41:25,160 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,170 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:25,170 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,170 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:25,170 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,170 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:25,170 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:25,170 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:25,170 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:25,376 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:25,376 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:25,376 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,376 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:25,376 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:25,376 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:25,376 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:25,376 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,414 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:41:25,414 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 08:41:25,416 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:25,416 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement] 08:41:25,420 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:41:25,420 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320, primaryVersion=13] 08:41:25,426 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 08:41:25,426 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320 for shard member-2-shard-cars-config_testModuleShardLeaderMovement 08:41:25,427 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:41:25,427 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-224670180] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,428 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#124707906 created and ready for shard:member-2-shard-cars-config_testModuleShardLeaderMovement 08:41:25,428 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#124707906] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,429 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 08:41:25,431 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:25,431 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testModuleShardLeaderMovement 08:41:25,431 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:41:25,432 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:25,434 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:41:25,434 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:41:25,434 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:25,434 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testModuleShardLeaderMovement is initialized 08:41:25,434 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,434 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:41:25,435 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 08:41:25,435 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-30T08:41:25.431966146Z 08:41:25,437 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 08:41:25,437 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 08:41:25,437 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:25,437 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:41:25,437 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@27bc6c5e 08:41:25,437 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from null to Follower 08:41:25,438 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,440 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-30T08:41:25.438119585Z 08:41:25,440 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 08:41:25,440 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:25,441 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-894543503] with name member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 08:41:25,441 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-894543503] 08:41:25,441 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 449.7 μs 08:41:25,441 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 08:41:25,441 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 08:41:25,441 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 127.9 μs 08:41:25,441 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-894543503] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,443 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:41:25,447 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@297b86d9 08:41:25,447 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,447 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320 08:41:25,448 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 08:41:25,448 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testModuleShardLeaderMovement 08:41:25,448 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 08:41:25,450 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:25,452 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 08:41:25,452 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,452 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:41:25,452 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:41:25,454 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#718051043] 08:41:25,454 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:25,454 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-224670180] 08:41:25,454 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-224670180] 08:41:25,455 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1102900526] 08:41:25,455 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:41:25,455 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 08:41:25,455 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:41:25,455 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 08:41:25,456 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:25,456 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement] 08:41:25,458 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:41:25,458 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-191031766, primaryVersion=13] 08:41:25,458 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:41:25,461 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-191031766 for shard member-2-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,462 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 08:41:25,463 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,463 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:41:25,463 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:25,465 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:41:25,466 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-30T08:41:25.463863133Z 08:41:25,467 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 08:41:25,467 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-1400733113] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,468 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#1198933164 created and ready for shard:member-2-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,468 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#1198933164] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,469 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:25,471 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:41:25,472 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:41:25,472 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testModuleShardLeaderMovement is initialized 08:41:25,472 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:25,472 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,472 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 08:41:25,472 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:41:25,474 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,474 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:25,474 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 08:41:25,474 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 08:41:25,474 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@68468ce2 08:41:25,475 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 08:41:25,475 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:25,478 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-30T08:41:25.475220405Z 08:41:25,478 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 08:41:25,478 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:25,478 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1758796278] with name member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 08:41:25,478 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1758796278] 08:41:25,478 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 470.3 μs 08:41:25,478 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 08:41:25,478 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 08:41:25,479 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 112.6 μs 08:41:25,479 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1758796278] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,480 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:41:25,482 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-191031766 08:41:25,483 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 08:41:25,483 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,483 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 08:41:25,485 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:25,488 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@327809e1 08:41:25,488 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:41:25,488 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:25,488 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 08:41:25,488 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#1102900526] 08:41:25,488 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 08:41:25,488 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,488 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:41:25,492 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-1400733113] 08:41:25,492 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#-1400733113] 08:41:25,492 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:41:25,492 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 08:41:25,493 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:25,493 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement] 08:41:25,496 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:41:25,496 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320, primaryVersion=13] 08:41:25,499 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320 for shard member-3-shard-cars-config_testModuleShardLeaderMovement 08:41:25,499 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 08:41:25,499 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,499 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-1092339657 created and ready for shard:member-3-shard-cars-config_testModuleShardLeaderMovement 08:41:25,500 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-1092339657] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,500 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:41:25,501 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 08:41:25,502 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-3-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:25,503 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testModuleShardLeaderMovement 08:41:25,504 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 08:41:25,505 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:25,505 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-3-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:41:25,505 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:41:25,505 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testModuleShardLeaderMovement is initialized 08:41:25,505 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:25,505 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,506 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 08:41:25,506 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 08:41:25,506 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from null to Follower 08:41:25,508 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]}}" message is greater than follower's term 0 - updating term 08:41:25,509 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-30T08:41:25.504943193Z 08:41:25,510 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 08:41:25,511 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@237fe3ce 08:41:25,511 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:25,511 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,511 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:41:25,514 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-3-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-30T08:41:25.511721159Z 08:41:25,514 AM [cluster-test-shard-dispatcher-27] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 08:41:25,514 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:25,515 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1120126983] with name member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 08:41:25,515 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1120126983] 08:41:25,515 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 470.5 μs 08:41:25,515 AM [cluster-test-shard-dispatcher-27] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 08:41:25,515 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 08:41:25,515 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 95.65 μs 08:41:25,515 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1120126983] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,517 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 08:41:25,520 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320 08:41:25,520 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 08:41:25,521 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testModuleShardLeaderMovement 08:41:25,521 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 08:41:25,521 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:25,523 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:41:25,523 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:41:25,525 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@29c1349c 08:41:25,525 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:41:25,525 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,525 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:41:25,525 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1689940554] 08:41:25,525 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 08:41:25,525 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,525 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:25,535 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] 08:41:25,536 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] 08:41:25,536 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-475863856] 08:41:25,536 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:41:25,536 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 08:41:25,536 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:41:25,536 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 08:41:25,537 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:25,537 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement] 08:41:25,539 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:41:25,539 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-191031766, primaryVersion=13] 08:41:25,541 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 08:41:25,541 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-501289331] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,541 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-191031766 for shard member-3-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,541 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#2026672781 created and ready for shard:member-3-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,542 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:41:25,542 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#2026672781] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,543 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,544 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 08:41:25,544 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 08:41:25,544 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:25,547 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-30T08:41:25.544066089Z 08:41:25,547 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:25,550 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:41:25,552 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:25,552 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:41:25,552 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,552 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testModuleShardLeaderMovement is initialized 08:41:25,552 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 08:41:25,552 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]}}" message is greater than follower's term 0 - updating term 08:41:25,555 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,555 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 08:41:25,555 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:25,555 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 08:41:25,555 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@37b2d490 08:41:25,555 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 08:41:25,555 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:25,558 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-3-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2025-08-30T08:41:25.555752660Z 08:41:25,558 AM [cluster-test-shard-dispatcher-28] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 08:41:25,558 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:25,559 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1797733466] with name member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 08:41:25,559 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1797733466] 08:41:25,559 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 505.5 μs 08:41:25,559 AM [cluster-test-shard-dispatcher-28] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 08:41:25,559 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 08:41:25,559 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 64.04 μs 08:41:25,559 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1797733466] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,560 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 08:41:25,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-191031766 08:41:25,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 08:41:25,565 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:25,565 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,565 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 08:41:25,565 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 08:41:25,566 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 08:41:25,567 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 08:41:25,567 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,568 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@53553c7f 08:41:25,568 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:41:25,568 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:41:25,568 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-475863856] 08:41:25,568 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done true 08:41:25,568 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:41:25,568 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-501289331] 08:41:25,568 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-501289331] 08:41:25,569 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#607822915] 08:41:25,569 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:25,569 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320] 08:41:25,569 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320] 08:41:25,569 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#718051043] 08:41:25,569 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:25,569 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-224670180] 08:41:25,569 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-224670180] 08:41:25,570 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1689940554] 08:41:25,570 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:25,570 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] 08:41:25,570 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] 08:41:25,571 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-1 for shard ShardName{value=cars}, datastoreType Config 08:41:25,571 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#607822915] 08:41:25,572 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:25,572 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320] 08:41:25,573 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320] 08:41:25,573 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 08:41:25,573 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 08:41:25,575 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#607822915] 08:41:25,575 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:25,575 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320] 08:41:25,575 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320] 08:41:25,579 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#718051043] 08:41:25,579 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:25,579 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-224670180] 08:41:25,579 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-224670180] 08:41:25,579 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1689940554] 08:41:25,579 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:25,579 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] 08:41:25,579 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] 08:41:25,580 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-2 for shard ShardName{value=cars}, datastoreType Config 08:41:25,580 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#718051043] 08:41:25,580 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:25,580 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-224670180] 08:41:25,580 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-224670180] 08:41:25,580 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 08:41:25,590 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@26668da9 08:41:25,590 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6df2fdf8 08:41:25,591 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6df2fdf8 08:41:25,591 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 08:41:25,593 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@21a77a25 08:41:25,593 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:41:25,593 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@b90eb24 08:41:25,595 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 08:41:25,596 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:41:25,600 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 2 08:41:25,600 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:41:25,600 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:25,600 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,600 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:41:25,600 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c13b058 08:41:25,600 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 08:41:25,602 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 2 in "RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:41:25,602 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 2 in "RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 1 - switching to Follower 08:41:25,610 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 2 08:41:25,610 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:25,610 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,610 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 08:41:25,610 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@73071f66 08:41:25,610 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 08:41:25,611 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 08:41:25,612 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:41:25,612 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 08:41:25,613 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:41:25,613 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2c4d455e 08:41:25,613 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 08:41:25,613 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,613 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:41:25,613 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@68bc7811 08:41:25,613 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,613 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 08:41:25,614 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 23.29 ms 08:41:25,614 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@be65756 08:41:25,614 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,614 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:41:25,617 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:41:25,622 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 08:41:25,622 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#718051043] 08:41:25,622 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:25,622 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-224670180] 08:41:25,622 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-224670180] 08:41:25,622 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:41:25,623 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#607822915] 08:41:25,623 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:25,623 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320] 08:41:25,623 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-78367320] 08:41:25,624 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1689940554] 08:41:25,624 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:25,624 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] 08:41:25,624 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] 08:41:25,624 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-3 for shard ShardName{value=cars}, datastoreType Config 08:41:25,624 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1689940554] 08:41:25,624 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:25,624 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] 08:41:25,624 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] 08:41:25,624 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 08:41:25,626 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@30aae145 08:41:25,626 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@768a5508 08:41:25,626 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@768a5508 08:41:25,626 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 08:41:25,628 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1867c962 08:41:25,628 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@290ceacd 08:41:25,629 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 08:41:25,634 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 3 08:41:25,634 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 3 08:41:25,634 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:25,634 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,634 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:41:25,634 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4d4c90a3 08:41:25,634 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 08:41:25,635 AM [cluster-test-shard-dispatcher-16] [INFO] Follower - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 3 in "RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 2 - updating term 08:41:25,636 AM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 3 in "RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 2 - switching to Follower 08:41:25,643 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 3 08:41:25,643 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:25,643 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:25,643 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:41:25,643 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,643 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 08:41:25,643 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6b132d53 08:41:25,643 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:41:25,644 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@42921482 08:41:25,644 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 08:41:25,644 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 08:41:25,644 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,644 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 08:41:25,645 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@61ce998c 08:41:25,645 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,646 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:41:25,646 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@24f21930 08:41:25,646 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,647 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-3-shard-cars-config_testModuleShardLeaderMovement 08:41:25,647 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-3-shard-cars-config_testModuleShardLeaderMovement in 20.76 ms 08:41:25,647 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:41:25,648 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 08:41:25,648 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1689940554] 08:41:25,648 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:41:25,649 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:25,649 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] 08:41:25,649 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#-119368480] 08:41:25,649 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:41:25,650 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 08:41:25,650 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testModuleShardLeaderMovement 08:41:25,650 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 08:41:25,651 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@38de7e6b 08:41:25,651 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a6501f5 08:41:25,651 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a6501f5 08:41:25,651 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 08:41:25,652 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@366bd3c2 08:41:25,653 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 08:41:25,654 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@655b581e 08:41:25,658 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 4 08:41:25,658 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 4 08:41:25,658 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:25,658 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,659 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:41:25,659 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5b023a2d 08:41:25,659 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 08:41:25,660 AM [cluster-test-shard-dispatcher-29] [INFO] AbstractLeader - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 3 - switching to Follower 08:41:25,660 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 3 - updating term 08:41:25,666 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 4 08:41:25,667 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:41:25,667 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:25,667 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 08:41:25,667 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@204ff72d 08:41:25,667 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 08:41:25,668 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 08:41:25,668 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:41:25,668 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 08:41:25,668 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:41:25,668 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@8b74469 08:41:25,668 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 08:41:25,668 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,669 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@21c3d39 08:41:25,669 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,669 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-1-shard-cars-config_testModuleShardLeaderMovement 08:41:25,669 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-1-shard-cars-config_testModuleShardLeaderMovement in 18.72 ms 08:41:25,669 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@33a9f591 08:41:25,669 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:41:25,669 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:41:25,670 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 08:41:25,671 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testModuleShardLeaderMovement: journal closed 08:41:25,671 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:41:25,671 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:41:25,672 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:41:25,672 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 08:41:25,673 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1689940554] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1689940554] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,673 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1689940554] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#1689940554] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,673 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,673 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 08:41:25,673 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:41:25,673 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 08:41:25,674 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testModuleShardLeaderMovement: journal closed 08:41:25,674 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:41:25,674 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:41:25,676 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:25,676 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:25,682 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:25,682 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:25,683 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:25,683 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:25,685 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:41:25,698 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 08:41:25,698 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testModuleShardLeaderMovement 08:41:25,699 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 08:41:25,700 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 08:41:25,702 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testModuleShardLeaderMovement: journal closed 08:41:25,703 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:41:25,703 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:41:25,703 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 08:41:25,704 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#718051043] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#718051043] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,704 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#718051043] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#718051043] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:25,704 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testModuleShardLeaderMovement 08:41:25,704 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 08:41:25,705 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 08:41:25,705 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testModuleShardLeaderMovement: journal closed 08:41:25,705 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:41:25,705 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:41:25,707 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:25,707 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:26,176 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:41:26,695 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [4896329034584688243] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:41:26,724 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [3369036403065304015] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:41:26,725 AM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:26,725 AM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:26,727 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:41:26.730] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:26,744 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 08:41:26,745 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testModuleShardLeaderMovement 08:41:26,746 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testModuleShardLeaderMovement 08:41:26,746 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:41:26,747 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@725075ba 08:41:26,747 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@725075ba 08:41:26,747 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 08:41:26,747 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@f8e7ebe 08:41:26,747 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@3967a8b9 08:41:26,747 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@3967a8b9 08:41:26,748 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Stopping Shard 08:41:26,749 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-1-shard-people-config_testModuleShardLeaderMovement: journal closed 08:41:31,277 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 08:41:31,277 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 08:41:31,277 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:41:31,278 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:31,278 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:31,278 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:31,278 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:31,283 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:31,433 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:34,312 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:35,905 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer aborted 08:41:35,905 AM [cluster-test-shard-dispatcher-28] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Failed to transfer leadership in 9.159 s 08:41:35,905 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In unpauseLeader 08:41:35,906 AM [cluster-test-shard-dispatcher-28] [WARN] Leader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 08:41:35,907 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 08:41:35,907 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:41:35,907 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 08:41:35,907 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 08:41:35,907 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 08:41:35,908 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testModuleShardLeaderMovement: journal closed 08:41:35,909 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:41:35,909 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:41:35,909 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 08:41:35,910 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testModuleShardLeaderMovement 08:41:35,911 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@411aa700 08:41:35,911 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@411aa700 08:41:35,911 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 08:41:35,911 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testModuleShardLeaderMovement 08:41:35,911 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@75951d8a 08:41:35,911 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@75951d8a 08:41:35,912 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:41:35,912 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Stopping Shard 08:41:35,912 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@36f9cd0d 08:41:35,912 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testModuleShardLeaderMovement: journal closed 08:41:39,223 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:41:39,535 AM [cluster-test-shard-dispatcher-16] [WARN] Leader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 08:41:39,536 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:41:39,536 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer aborted 08:41:39,536 AM [cluster-test-shard-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: Failed to transfer leadership in 3.625 s 08:41:39,536 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In unpauseLeader 08:41:39,536 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:41:39,537 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 08:41:39,537 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testModuleShardLeaderMovement: journal closed 08:41:39,538 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:41:39,538 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:41:39,540 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:39,540 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:39,543 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:41:39.546] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:39,572 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:39,590 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [1927992842131120014] 08:41:39,591 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:41:39,594 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:41:39,594 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:41:39,596 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:39,596 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:39,611 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:41:39,611 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:41:39,611 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:41:39,613 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:39,614 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:39,617 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 08:41:39,626 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2106117017] 08:41:39,628 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2106117017] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,629 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:41:39,629 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:41:39,629 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:41:39,629 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardReplica 08:41:39,629 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardReplica 08:41:39,630 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:39,630 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:39,630 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:39,637 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:41:39,639 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@3c60bae0} 08:41:39,640 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} started 08:41:39,640 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:39,640 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardReplica 08:41:39,640 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:39,641 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:39,641 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:39,641 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica on shard member-1-shard-cars-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1286041048] 08:41:39,641 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardReplica 08:41:39,641 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:39,642 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:39,642 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica on shard member-1-shard-people-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#-1542233273] 08:41:39,642 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:39,642 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 08:41:39,643 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:39,644 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 08:41:39,645 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 08:41:39,645 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 08:41:39,645 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:39,646 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 08:41:39,647 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1695516667] 08:41:39,647 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:41:39,648 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:41:39,648 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:41:39,648 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardReplica 08:41:39,648 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardReplica 08:41:39,649 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:39,649 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:39,649 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:39,652 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1286041048] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,652 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#-1542233273] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,652 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1695516667] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,652 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#-181463393 created and ready for shard:member-1-shard-cars-config_testRemoveShardReplica 08:41:39,652 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#-181463393] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,652 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#-1997651368 created and ready for shard:member-1-shard-people-config_testRemoveShardReplica 08:41:39,652 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#-1997651368] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,654 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:39,655 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:39,659 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:39,659 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 08:41:39,659 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:39,659 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:41:39,660 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardReplica is initialized 08:41:39,660 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:41:39,660 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardReplica is initialized 08:41:39,660 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:39,660 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@60148fec} 08:41:39,660 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:39,660 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from null to Follower 08:41:39,660 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 08:41:39,660 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:41:39,661 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:39,661 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardReplica 08:41:39,664 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:39,664 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:39,664 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 08:41:39,664 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:41:39,664 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 08:41:39,664 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#1987235450] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,665 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#-1419028262 created and ready for shard:member-1-shard-cars-oper_testRemoveShardReplica 08:41:39,665 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#-1419028262] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,665 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:39,666 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 08:41:39,667 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from null to Follower 08:41:39,667 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:39,667 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from null to Follower 08:41:39,668 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:39,668 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:39,668 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-1-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#1987235450] 08:41:39,668 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardReplica 08:41:39,669 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:39,669 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:39,669 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-1-shard-people-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#215030880] 08:41:39,669 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 08:41:39,669 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#215030880] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,670 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#-1149004565 created and ready for shard:member-1-shard-people-oper_testRemoveShardReplica 08:41:39,670 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:39,670 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#-1149004565] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,671 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:41:39,671 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardReplica is initialized 08:41:39,671 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:39,671 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:39,671 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 08:41:39,671 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:41:39,671 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:39,671 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from null to Follower 08:41:39,672 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 08:41:39,673 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:39,675 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:39,676 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:39,676 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:39,676 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 08:41:39,676 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:41:39,676 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardReplica is initialized 08:41:39,676 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:41:39,676 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from null to Follower 08:41:39,680 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:39,691 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [527959802838395833] 08:41:39,691 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.1.5] ... 08:41:39,697 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:39,697 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:41:39,708 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:39,708 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:39,722 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:39,733 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:39,740 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 08:41:39,754 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#77599082] 08:41:39,756 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:41:39,756 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:41:39,756 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:41:39,756 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 08:41:39,756 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 08:41:39,759 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#77599082] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,763 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:41:39,764 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@36442e30} 08:41:39,764 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} started 08:41:39,765 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:39,765 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 08:41:39,765 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:39,767 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:39,769 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 08:41:39,773 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 08:41:39,774 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 08:41:39,774 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-448144133] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,775 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:39,776 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#1877136823] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,776 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#1877136823 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 08:41:39,777 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#336818421] 08:41:39,777 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#336818421] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,778 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 08:41:39,779 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#775505327] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,780 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 08:41:39,781 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:39,781 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 08:41:39,782 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-1117674324] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,782 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 08:41:39,783 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 08:41:39,781 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:39,784 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:41:39,784 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:41:39,784 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:41:39,784 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 08:41:39,784 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 08:41:39,785 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@6fd7860d} 08:41:39,785 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:39,785 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 08:41:39,786 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#-784123581] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,786 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 08:41:39,787 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#1566998512 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 08:41:39,787 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#1566998512] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,787 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:39,788 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 08:41:39,788 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:39,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:39,789 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:39,789 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 08:41:39,789 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:41:39,789 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 08:41:39,789 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:41:39,790 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 08:41:39,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 08:41:39,790 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#389031225] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,791 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 08:41:39,791 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#131098633 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 08:41:39,791 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#131098633] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,791 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:39,792 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 08:41:39,796 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:39,796 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:39,797 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:39,797 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-1117674324 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 08:41:39,799 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:39,799 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:41:39,799 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 08:41:39,799 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:39,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:41:39,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 08:41:39,800 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:39,800 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:39,800 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:39,800 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:39,800 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:39,800 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:41:39,800 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 08:41:39,801 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 08:41:39,801 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:41:39,801 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 08:41:39,801 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:41:39,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 08:41:39,801 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:39,801 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:39,801 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 08:41:39,801 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:41:39,801 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 08:41:39,823 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 08:41:39,826 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:39,830 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:41:39,834 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:41:39,835 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:39,835 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:39,835 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:41:39,835 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:39,835 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:41:39,835 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:39,835 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:39,835 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:41:39,835 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:39,835 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:39,835 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:41:39,835 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:39,835 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:39,835 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:39,850 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-5732558586834721404] 08:41:39,851 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.1.5] ... 08:41:39,855 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:39,855 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:41:39,857 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:39,858 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:39,873 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:39,875 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:39,882 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 08:41:39,887 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-684514966] 08:41:39,888 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:41:39,888 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-684514966] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,889 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:41:39,889 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:41:39,889 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardReplica 08:41:39,889 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardReplica 08:41:39,898 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:41:39,901 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@358ecc6c} 08:41:39,901 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} started 08:41:39,902 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:39,902 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardReplica 08:41:39,902 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:39,903 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:41:39,903 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:39,906 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:41:39,906 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 08:41:39,911 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardReplica 08:41:39,912 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#161261857] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,912 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 08:41:39,912 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-1902392412 created and ready for shard:member-3-shard-cars-config_testRemoveShardReplica 08:41:39,912 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#-1902392412] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,913 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-3-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:39,914 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:39,914 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:39,914 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:41:39,914 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:39,914 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:41:39,914 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 08:41:39,914 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica#-1892715832] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,915 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1565332442] 08:41:39,915 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 08:41:39,915 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:41:39,915 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:41:39,915 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:41:39,915 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardReplica 08:41:39,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardReplica 08:41:39,916 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:39,916 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:39,916 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:39,917 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#1786275100 created and ready for shard:member-3-shard-people-config_testRemoveShardReplica 08:41:39,917 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:39,917 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:39,919 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1565332442] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,919 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#1786275100] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,920 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:39,921 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:41:39,921 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardReplica is initialized 08:41:39,921 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:39,921 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:39,921 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 08:41:39,921 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:39,921 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:41:39,921 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardReplica from null to Follower 08:41:39,923 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 08:41:39,924 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@7a742e96} 08:41:39,924 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 08:41:39,925 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:39,925 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardReplica 08:41:39,925 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:41:39,925 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:39,925 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:41:39,925 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-3-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#-1256478705] 08:41:39,925 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardReplica 08:41:39,926 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 08:41:39,926 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:39,926 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:41:39,926 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-3-shard-people-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#-1808105829] 08:41:39,927 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 08:41:39,927 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:39,927 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-3-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:39,928 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 08:41:39,928 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#-1256478705] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,928 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#262641035 created and ready for shard:member-3-shard-people-oper_testRemoveShardReplica 08:41:39,928 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#1282248324 created and ready for shard:member-3-shard-cars-oper_testRemoveShardReplica 08:41:39,929 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#-1808105829] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,929 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#262641035] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,929 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#1282248324] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:39,931 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:39,932 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 08:41:39,933 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:39,933 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 08:41:39,934 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:39,935 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:39,935 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:41:39,935 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardReplica is initialized 08:41:39,935 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:39,935 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:39,935 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 08:41:39,935 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:39,935 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:39,936 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:41:39,936 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardReplica from null to Follower 08:41:39,936 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:41:39,936 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardReplica is initialized 08:41:39,936 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:39,936 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:39,936 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 08:41:39,937 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:39,937 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:41:39,937 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardReplica from null to Follower 08:41:39,937 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:39,938 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:41:39,938 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardReplica is initialized 08:41:39,938 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:39,938 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:39,938 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , received role change from null to Follower 08:41:39,938 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:39,938 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:41:39,939 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardReplica from null to Follower 08:41:40,12 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 1 08:41:40,12 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:40,12 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:40,12 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:40,12 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Follower to Candidate 08:41:40,12 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Follower to Candidate 08:41:40,32 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 1 08:41:40,32 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:40,32 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:40,32 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:40,33 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Follower to Candidate 08:41:40,33 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Follower to Candidate 08:41:40,40 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 1 08:41:40,41 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:40,41 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:40,41 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:40,41 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Follower to Candidate 08:41:40,41 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Follower to Candidate 08:41:40,81 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 1 08:41:40,81 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:40,81 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:40,81 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:40,82 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Follower to Candidate 08:41:40,82 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Follower to Candidate 08:41:40,402 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 2 08:41:40,431 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 2 08:41:40,452 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 2 08:41:40,501 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 2 08:41:40,610 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:41:40,610 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:41:40,612 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:40,612 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:41:40,612 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:40,612 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:41:40,612 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:41:40,612 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:40,612 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:41:40,612 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:40,612 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:40,612 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:41:40,612 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:40,613 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardReplica: Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:41:40,612 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:41:40,613 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:40,622 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:40,622 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:40,622 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:41:40,622 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:40,622 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:41:40,623 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:41:40,623 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:41:40,623 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:41:40,623 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:41:40,738 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:40,738 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:40,738 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:41:40,738 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:41:40,738 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:41:40,738 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:41:40,738 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:40,738 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:41:40,738 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:41:40,738 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:41:40,738 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:41:40,738 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:41:40,739 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:41:40,738 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:41:40,739 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:41:40,739 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:41:40,739 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:41:40,739 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:40,739 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:41:40,739 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:41:40,739 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:41:40,739 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:41:40,739 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:41:40,739 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardReplica: Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:41:40,782 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 3 08:41:40,785 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:40,785 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:40,794 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:40,794 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:40,794 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: became leader with frontend state for [] 08:41:40,794 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Candidate to Leader 08:41:40,795 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4a1a1a16 08:41:40,795 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Candidate to Leader 08:41:40,796 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@11cbedce 08:41:40,797 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done false 08:41:40,797 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 08:41:40,797 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3100f4ab 08:41:40,832 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 3 08:41:40,834 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:40,834 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:40,850 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:40,850 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:40,850 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: became leader with frontend state for [] 08:41:40,850 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Candidate to Leader 08:41:40,850 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@78d571d1 08:41:40,851 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Candidate to Leader 08:41:40,852 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@60fa922b 08:41:40,852 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 08:41:40,852 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 3 08:41:40,853 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done false 08:41:40,853 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@e3676cf 08:41:40,855 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:40,855 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:40,862 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:40,862 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:40,862 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: became leader with frontend state for [] 08:41:40,862 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Candidate to Leader 08:41:40,862 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@44ddfd25 08:41:40,862 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Candidate to Leader 08:41:40,862 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:41:40,863 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@769f9b2f 08:41:40,863 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:41:40,863 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 08:41:40,863 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@42a39a3a 08:41:40,863 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:41:40,863 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done false 08:41:40,878 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:41:40,881 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 3 08:41:40,884 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:40,884 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:40,890 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:40,890 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:40,890 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: became leader with frontend state for [] 08:41:40,890 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Candidate to Leader 08:41:40,890 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@40e27166 08:41:40,891 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Candidate to Leader 08:41:40,891 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 08:41:40,891 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 08:41:40,891 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:41:40,891 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 08:41:40,892 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@59922ffb 08:41:40,892 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3bfd17ac 08:41:40,892 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardReplica is ready 08:41:40,892 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardReplica status sync done false 08:41:40,892 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 08:41:40,892 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 08:41:40,892 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2106117017] 08:41:40,892 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:40,892 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1286041048] 08:41:40,893 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1286041048] 08:41:40,893 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#77599082] 08:41:40,893 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:40,893 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-448144133] 08:41:40,894 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-448144133] 08:41:40,894 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-684514966] 08:41:40,894 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:40,894 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#161261857] 08:41:40,894 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#161261857] 08:41:40,896 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-3}, datastoreType Config 08:41:40,897 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 08:41:40,899 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:40,899 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, primaryVersion=13] 08:41:40,900 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica for shard member-3-shard-cars-config_testRemoveShardReplica 08:41:40,907 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 08:41:40,907 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 08:41:40,907 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 08:41:40,909 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2106117017] 08:41:40,909 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:40,909 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1286041048] 08:41:40,909 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1286041048] 08:41:40,910 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:40,910 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#77599082] 08:41:40,910 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 08:41:40,911 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:40,911 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-448144133] 08:41:40,911 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#-448144133] 08:41:40,911 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-684514966] 08:41:40,911 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:40,911 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#161261857] 08:41:40,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#161261857] with 27000 ms timeout 08:41:40,916 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Stopping Shard 08:41:40,916 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardReplica: journal closed 08:41:40,918 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 08:41:40,918 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [people] 08:41:40,918 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#161261857] 08:41:40,961 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-684514966] 08:41:40,962 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:40,962 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [people] 08:41:40,962 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:41:40,963 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardReplica 08:41:40,963 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 08:41:40,963 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:41:40,964 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Stopping Shard 08:41:40,964 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Stopping Shard 08:41:40,964 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2559] as [Down] 08:41:40,965 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal closed 08:41:40,965 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal closed 08:41:40,966 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:41:40,966 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:41:40,967 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 08:41:40,968 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 08:41:40,968 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 08:41:40,968 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:41:40,968 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Stopping Shard 08:41:40,968 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Stopping Shard 08:41:40,969 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal closed 08:41:40,969 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal closed 08:41:40,969 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:41:40,969 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:41:40,971 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 08:41:40,973 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Node has been marked as DOWN. Shutting down myself 08:41:40,973 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:40,973 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:40,976 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Shutting down... 08:41:40,976 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Successfully shut down 08:41:40,983 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:40,984 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:40,985 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:40,985 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:40,988 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:41:41,7 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 08:41:41,23 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [1787423626276754234] 08:41:41,23 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.1.5] ... 08:41:41,24 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Registered cluster JMX MBean [pekko:type=Cluster] 08:41:41,24 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:41:41,24 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:41,31 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:41,33 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:41,35 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 08:41:41,38 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:41,41 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#262347091] 08:41:41,42 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#262347091] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:41,42 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:41:41,42 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:41:41,42 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:41:41,42 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 08:41:41,43 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 08:41:41,54 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@1bd0322a} 08:41:41,54 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} started 08:41:41,54 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:41,56 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:41,58 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 08:41:41,59 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-723095387] 08:41:41,60 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:41:41,60 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:41:41,60 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:41:41,60 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 08:41:41,60 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 08:41:41,61 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:41,61 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 08:41:41,61 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-723095387] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:41,62 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 08:41:41,63 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 08:41:41,63 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 08:41:41,63 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#846194735] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:41,63 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:41,64 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#-1018873607] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:41,64 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 08:41:41,65 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#598324315 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 08:41:41,66 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:41,68 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#598324315] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:41,68 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:41,69 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:41,70 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 08:41:41,70 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:41,70 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:41,70 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:41:41,70 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 08:41:41,71 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#10167704 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 08:41:41,71 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 08:41:41,71 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:41,71 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:41:41,71 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#10167704] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:41,72 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 08:41:41,73 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@15beda3d} 08:41:41,74 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} started 08:41:41,74 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:41,74 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 08:41:41,74 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:41,74 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 08:41:41,74 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 08:41:41,74 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:41,74 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:41,74 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:41:41,74 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 08:41:41,75 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 08:41:41,75 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:41,76 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 08:41:41,76 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#1486966650] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:41,76 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#-1622715423] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:41,76 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 08:41:41,76 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:41:41,76 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#319029661 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 08:41:41,77 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 08:41:41,77 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-1396906080 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 08:41:41,77 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:41:41,77 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 08:41:41,77 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 08:41:41,78 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:41,79 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#319029661] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:41,79 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-1396906080] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:41,80 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:41,81 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:41,81 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:41:41,81 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 08:41:41,81 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:41,81 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:41,82 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 08:41:41,82 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:41:41,82 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 08:41:41,82 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:41,83 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:41:41,83 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 08:41:41,83 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:41,83 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:41,83 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 08:41:41,84 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:41:41,84 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 08:41:41,111 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done true 08:41:41,113 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 314, lastApplied : -1, commitIndex : -1 08:41:41,170 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done true 08:41:41,171 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 08:41:41,180 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done true 08:41:41,181 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:41:41,433 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:41,492 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:41,501 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:41,629 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:41:41,751 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:41,812 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:41,822 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:41,990 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [527959802838395833] but current is [1787423626276754234]. Reason: ActorSystem terminated 08:41:41,997 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [527959802838395833] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:41:42,32 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#648087149]] intended for system uid [527959802838395833], self uid is [1787423626276754234] 08:41:42,33 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-people-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 08:41:42,34 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-people-oper_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 08:41:42,34 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 08:41:42,34 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=0, prevLogTerm=3, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 08:41:42,36 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:41:42,37 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1214ede5 08:41:42,37 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 08:41:42,37 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done true 08:41:42,38 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): The followers log is empty and the senders prevLogIndex is 0 08:41:42,38 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 08:41:42,38 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 08:41:42,38 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6e01aa4c 08:41:42,38 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2b0339e8 08:41:42,38 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done true 08:41:42,38 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): The followers log is empty and the senders prevLogIndex is 0 08:41:42,38 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2d813a8e 08:41:42,38 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 08:41:42,38 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): The followers log is empty and the senders prevLogIndex is 0 08:41:42,38 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 08:41:42,38 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 08:41:42,38 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done true 08:41:42,38 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 08:41:42,38 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 08:41:42,40 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:41:42,40 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1187, lastApplied : -1, commitIndex : -1 08:41:42,41 AM [cluster-test-shard-dispatcher-20] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1242, lastApplied : -1, commitIndex : -1 08:41:42,41 AM [cluster-test-shard-dispatcher-18] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1177, lastApplied : -1, commitIndex : -1 08:41:42,41 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1134, lastApplied : 0, commitIndex : 0 08:41:42,41 AM [cluster-test-shard-dispatcher-10] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 08:41:42,41 AM [cluster-test-shard-dispatcher-10] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): follower member-2-shard-cars-config_testRemoveShardReplica appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 08:41:42,41 AM [cluster-test-shard-dispatcher-10] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 08:41:42,42 AM [cluster-test-shard-dispatcher-10] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): follower member-2-shard-cars-config_testRemoveShardReplica appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 08:41:42,42 AM [cluster-test-shard-dispatcher-10] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 08:41:42,42 AM [cluster-test-shard-dispatcher-10] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): follower member-2-shard-cars-config_testRemoveShardReplica appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 08:41:42,43 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:42,74 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 08:41:42,131 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:42,140 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:42,391 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:42,392 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:41:42,394 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 352, lastApplied : -1, commitIndex : -1 08:41:42,450 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:42,451 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:42,451 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 410, lastApplied : -1, commitIndex : -1 08:41:42,459 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:42,460 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 419, lastApplied : -1, commitIndex : -1 08:41:42,461 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:42,491 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 447, lastApplied : 0, commitIndex : 0 08:41:42,712 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:42,712 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:42,772 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:42,772 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:42,781 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:42,781 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:42,811 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 08:41:43,31 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:43,32 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:43,56 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:41:43,92 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:43,93 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:43,100 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:43,100 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:43,131 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 08:41:43,352 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:43,352 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:43,412 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:43,412 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:43,421 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:43,422 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:41:43,451 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:41:43,651 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#648087149]] intended for system uid [527959802838395833], self uid is [1787423626276754234] 08:41:43,671 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:43,672 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:43,731 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:43,732 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:43,741 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:43,742 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:43,771 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 08:41:43,919 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#88732954]] intended for system uid [527959802838395833], self uid is [1787423626276754234] 08:41:43,992 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:43,992 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:41:44,51 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:44,51 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:44,60 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:44,60 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:44,76 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:41:44,91 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 08:41:44,313 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:44,314 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:41:44,371 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:44,371 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:44,381 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:44,381 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:44,410 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 08:41:44,633 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:44,633 AM [cluster-test-shard-dispatcher-10] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:44,690 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:44,692 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:41:44,700 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:44,700 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:44,731 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 08:41:44,954 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:44,954 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:41:45,13 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:45,14 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 323, lastApplied : -1, commitIndex : -1 08:41:45,20 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:45,20 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:45,50 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 08:41:45,97 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:41:45,272 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:41:45,272 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:45,332 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:45,332 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:45,340 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:45,340 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:45,370 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:41:45,592 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:45,599 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 326, lastApplied : -1, commitIndex : -1 08:41:45,651 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:45,652 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:45,660 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:45,660 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:45,670 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#648087149]] intended for system uid [527959802838395833], self uid is [1787423626276754234] 08:41:45,690 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:41:45,908 AM [cluster-test-shard-dispatcher-31] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 08:41:45,909 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:41:45,909 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:45,909 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 08:41:45,909 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 08:41:45,912 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 08:41:45,912 AM [cluster-test-shard-dispatcher-31] [INFO] IsolatedLeader - member-1-shard-cars-oper_testRemoveShardReplica: IsolatedLeader member-1-shard-cars-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 08:41:45,913 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 08:41:45,913 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:45,913 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 08:41:45,913 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from IsolatedLeader to Leader 08:41:45,913 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:41:45,913 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 314, lastApplied : -1, commitIndex : -1 08:41:45,969 AM [cluster-test-shard-dispatcher-19] [WARN] Leader - member-1-shard-people-config_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 08:41:45,969 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:41:45,969 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:45,969 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 08:41:45,970 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 08:41:45,972 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:45,972 AM [cluster-test-shard-dispatcher-19] [INFO] IsolatedLeader - member-1-shard-people-config_testRemoveShardReplica: IsolatedLeader member-1-shard-people-config_testRemoveShardReplica switching from IsolatedLeader to Leader 08:41:45,972 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 08:41:45,972 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:45,972 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 08:41:45,972 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from IsolatedLeader to Leader 08:41:45,972 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 08:41:45,974 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:41:45,979 AM [cluster-test-shard-dispatcher-19] [WARN] Leader - member-1-shard-people-oper_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 08:41:45,979 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:41:45,979 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:45,979 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 08:41:45,979 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 08:41:45,981 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:45,981 AM [cluster-test-shard-dispatcher-19] [INFO] IsolatedLeader - member-1-shard-people-oper_testRemoveShardReplica: IsolatedLeader member-1-shard-people-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 08:41:45,981 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 08:41:45,981 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:45,981 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:41:45,982 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 08:41:45,982 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from IsolatedLeader to Leader 08:41:45,982 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:41:46,8 AM [cluster-test-shard-dispatcher-19] [WARN] Leader - member-1-shard-cars-config_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 08:41:46,8 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:41:46,8 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:46,9 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 08:41:46,9 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 08:41:46,10 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 08:41:46,11 AM [cluster-test-shard-dispatcher-19] [INFO] IsolatedLeader - member-1-shard-cars-config_testRemoveShardReplica: IsolatedLeader member-1-shard-cars-config_testRemoveShardReplica switching from IsolatedLeader to Leader 08:41:46,11 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 08:41:46,11 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:46,11 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 08:41:46,11 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from IsolatedLeader to Leader 08:41:46,11 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 08:41:46,117 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:41:46,231 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 08:41:46,231 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 08:41:46,291 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : -1, commitIndex : -1 08:41:46,292 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 08:41:46,300 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 08:41:46,301 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:41:46,331 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:41:46,552 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:41:46,552 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:41:46,611 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:46,612 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:46,620 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:46,620 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:46,651 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 08:41:46,729 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Down)]. 08:41:46,731 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:46,731 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:46,871 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:46,872 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:46,932 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:46,932 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:46,941 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:46,941 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:46,971 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 08:41:46,998 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Down)]. 08:41:46,999 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:46,999 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:47,137 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:41:47,191 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:47,191 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:47,252 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:47,252 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:47,261 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:47,261 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:47,290 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 08:41:47,511 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:47,511 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:47,571 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:47,572 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:47,580 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:47,580 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:47,611 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:41:47,832 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:47,832 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:41:47,891 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:47,892 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:47,900 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:47,901 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:47,931 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:41:47,959 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#88732954]] intended for system uid [527959802838395833], self uid is [1787423626276754234] 08:41:48,19 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2559] 08:41:48,20 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:48,21 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:48,21 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [527959802838395833] but current is [1787423626276754234]. Reason: Cluster member removed, previous status [Down] 08:41:48,119 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:48,119 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:48,120 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [527959802838395833] but current is [1787423626276754234]. Reason: Cluster member removed, previous status [Down] 08:41:48,151 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:48,151 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:41:48,156 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:41:48,159 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:41:48,161 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:48,161 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:41:48,161 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:48,161 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:48,161 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 08:41:48,161 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:48,161 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:48,161 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:41:48,162 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:48,162 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:41:48,162 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 08:41:48,162 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:48,162 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:48,162 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:41:48,162 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:48,162 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} is now ready 08:41:48,162 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#262347091] 08:41:48,162 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:48,162 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#846194735] 08:41:48,162 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#846194735] 08:41:48,163 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-2}, datastoreType Config 08:41:48,163 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-2}] 08:41:48,163 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:48,164 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:41:48,164 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:41:48,164 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:48,164 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:48,164 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1286041048, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4d3f38c8] 08:41:48,165 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1286041048 for shard member-2-shard-cars-config_testRemoveShardReplica 08:41:48,166 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 08:41:48,166 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 08:41:48,166 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 08:41:48,166 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-2106117017] 08:41:48,166 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:48,166 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1286041048] 08:41:48,167 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#-1286041048] 08:41:48,167 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#262347091] 08:41:48,167 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:48,167 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:48,168 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#846194735] 08:41:48,168 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#846194735] with 27000 ms timeout 08:41:48,168 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2 : Local Shard replica for shard cars has been removed 08:41:48,168 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 08:41:48,169 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Stopping Shard 08:41:48,169 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal closed 08:41:48,170 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#846194735] 08:41:48,211 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:48,211 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:41:48,218 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#262347091] 08:41:48,218 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:48,218 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Local shard cars not found - shards present: [people] 08:41:48,219 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 08:41:48,219 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 08:41:48,219 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 08:41:48,221 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Stopping Shard 08:41:48,221 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:48,221 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal closed 08:41:48,221 AM [cluster-test-shard-dispatcher-19] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:41:48,222 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:41:48,222 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:41:48,222 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} 08:41:48,223 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 08:41:48,223 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 08:41:48,223 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:41:48,223 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Stopping Shard 08:41:48,223 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Stopping Shard 08:41:48,224 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal closed 08:41:48,224 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal closed 08:41:48,224 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:41:48,224 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:41:48,226 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:48,226 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:48,231 AM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:48,232 AM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:48,232 AM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:48,232 AM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:48,234 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:41:48,245 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:41:48,247 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardReplica 08:41:48,247 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 08:41:48,248 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Stopping Shard 08:41:48,248 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardReplica: journal closed 08:41:48,249 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:41:48,249 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:41:48,249 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 08:41:48,250 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardReplica 08:41:48,250 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardReplica 08:41:48,250 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:41:48,250 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Stopping Shard 08:41:48,250 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Stopping Shard 08:41:48,250 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardReplica: journal closed 08:41:48,251 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardReplica: journal closed 08:41:48,251 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:41:48,251 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:41:48,253 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:48,253 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:49,38 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:41:49,43 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:49,43 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:49,43 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:41:49,43 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:41:49,43 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:41:49,43 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:41:49,43 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:41:49,43 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:41:49,43 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:41:49,43 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:41:49,43 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:41:49,43 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 08:41:49,43 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:41:49,43 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardReplica: Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:41:49,43 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:41:49,238 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [1787423626276754234] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:41:49,269 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-5732558586834721404] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:41:49,271 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:49,272 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:49,276 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:41:49.278] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:49,287 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:41:49,289 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveShardReplica 08:41:49,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardReplica 08:41:49,290 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:41:49,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@46ea9334 08:41:49,290 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@46ea9334 08:41:49,290 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Stopping Shard 08:41:49,290 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4900a5b7 08:41:49,290 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@66fd200a 08:41:49,290 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@66fd200a 08:41:49,290 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 08:41:49,292 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardReplica: journal closed 08:41:49,808 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: leader transfer aborted 08:41:49,808 AM [cluster-test-shard-dispatcher-31] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: Failed to transfer leadership in 518.5 ms 08:41:49,809 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In unpauseLeader 08:41:49,809 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Stopping Shard 08:41:49,810 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardReplica: journal closed 08:41:49,811 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:41:49,811 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:41:49,811 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 08:41:49,812 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardReplica 08:41:49,812 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardReplica 08:41:49,812 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:41:49,812 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@c0232f3 08:41:49,813 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@9095b42 08:41:49,813 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2739151b 08:41:49,813 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@11d8aeb3 08:41:49,813 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2739151b 08:41:49,813 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@11d8aeb3 08:41:49,813 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 08:41:49,813 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 08:41:50,139 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: leader transfer aborted 08:41:50,139 AM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: Failed to transfer leadership in 326.3 ms 08:41:50,139 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In unpauseLeader 08:41:50,140 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Stopping Shard 08:41:50,140 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardReplica: journal closed 08:41:50,388 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer aborted 08:41:50,388 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: Failed to transfer leadership in 575.7 ms 08:41:50,388 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In unpauseLeader 08:41:50,389 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Stopping Shard 08:41:50,390 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardReplica: journal closed 08:41:50,390 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:41:50,390 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:41:50,392 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:50,393 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:50,396 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:41:50.400] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:50,423 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:50,434 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-3622220259111808882] 08:41:50,435 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:41:50,437 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:41:50,437 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:41:50,443 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:50,443 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:50,456 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:50,457 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:41:50,457 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:41:50,457 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:41:50,459 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:50,463 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 08:41:50,467 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#667600251] 08:41:50,468 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:41:50,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:41:50,468 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:41:50,468 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:50,469 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveAllShardReplicas 08:41:50,469 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:50,469 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:50,469 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:41:50,469 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#667600251] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,479 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:41:50,481 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@151e8fc8} 08:41:50,481 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 08:41:50,482 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:50,484 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:50,487 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:50,488 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:50,489 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:50,489 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:50,489 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-1-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1325220282] 08:41:50,489 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveAllShardReplicas 08:41:50,490 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 08:41:50,491 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:41:50,491 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:41:50,491 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-1-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-1395913076] 08:41:50,492 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:50,492 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:50,493 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-1395913076] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,493 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1325220282] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,493 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#1076331263 created and ready for shard:member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:50,493 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#-107435635] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,494 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1375667886] 08:41:50,494 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:41:50,494 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:41:50,494 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:41:50,494 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,495 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:50,497 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:50,498 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:50,498 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:50,498 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#-107435635 created and ready for shard:member-1-shard-people-config_testRemoveAllShardReplicas 08:41:50,499 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,499 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:50,499 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:50,500 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:50,501 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:50,501 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:50,502 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#1076331263] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,502 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1375667886] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,504 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 08:41:50,505 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@530bdf1b} 08:41:50,505 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:50,506 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,506 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,506 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 08:41:50,506 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:41:50,506 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveAllShardReplicas is initialized 08:41:50,506 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:41:50,506 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 08:41:50,507 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:50,507 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,510 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:50,510 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,510 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,511 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:50,511 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:50,512 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,512 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,512 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-1-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1445651546] 08:41:50,512 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:50,512 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from null to Follower 08:41:50,512 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:41:50,512 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveAllShardReplicas is initialized 08:41:50,512 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:50,512 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:50,512 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-1-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#-2032997989] 08:41:50,512 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:50,513 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1445651546] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,513 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#-121645648 created and ready for shard:member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,513 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#-121645648] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,513 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#-2032997989] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,513 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:50,514 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#-1983004413 created and ready for shard:member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:50,514 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#-1983004413] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,514 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:50,515 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:50,518 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 08:41:50,518 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:41:50,518 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from null to Follower 08:41:50,519 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:50,519 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:50,520 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:50,521 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:50,522 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:41:50,522 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveAllShardReplicas is initialized 08:41:50,522 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,522 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,522 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 08:41:50,522 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 08:41:50,522 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from null to Follower 08:41:50,524 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:50,525 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:41:50,525 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveAllShardReplicas is initialized 08:41:50,525 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,525 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,525 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 08:41:50,525 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 08:41:50,525 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 08:41:50,533 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [4867104681791588656] 08:41:50,533 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.1.5] ... 08:41:50,535 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:50,535 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:41:50,541 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:50,541 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:50,560 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:50,563 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:50,568 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 08:41:50,576 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1228210354] 08:41:50,577 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1228210354] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,577 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:41:50,578 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:41:50,578 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:41:50,578 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveAllShardReplicas 08:41:50,578 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveAllShardReplicas 08:41:50,586 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:41:50,592 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@2ada0e44} 08:41:50,592 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 08:41:50,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:50,593 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveAllShardReplicas 08:41:50,593 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:50,594 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:50,598 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveAllShardReplicas 08:41:50,603 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:50,604 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:50,605 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 08:41:50,605 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:50,606 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#-1723342739 created and ready for shard:member-2-shard-cars-config_testRemoveAllShardReplicas 08:41:50,606 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#2014413535] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,606 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#-1723342739] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,607 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:50,607 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#1247411242] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,607 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#795810156] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,608 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:50,608 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1789905774] 08:41:50,608 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:50,609 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:41:50,609 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:41:50,609 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:41:50,609 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,609 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveAllShardReplicas 08:41:50,609 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:50,611 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:50,612 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#795810156 created and ready for shard:member-2-shard-people-config_testRemoveAllShardReplicas 08:41:50,612 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:41:50,612 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveAllShardReplicas is initialized 08:41:50,612 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,612 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,612 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 08:41:50,613 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:41:50,613 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:50,613 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1789905774] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,613 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveAllShardReplicas from null to Follower 08:41:50,615 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:50,615 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:41:50,615 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveAllShardReplicas is initialized 08:41:50,615 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,615 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,615 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 08:41:50,615 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:41:50,615 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveAllShardReplicas from null to Follower 08:41:50,624 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 08:41:50,625 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@30555307} 08:41:50,626 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 08:41:50,630 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:50,630 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,631 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveAllShardReplicas 08:41:50,631 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:50,632 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:50,633 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:50,634 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActor - member-2-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:50,634 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:41:50,635 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas#745427235] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,635 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas#90401091] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,635 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#848482290 created and ready for shard:member-2-shard-people-oper_testRemoveAllShardReplicas 08:41:50,635 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#712585193 created and ready for shard:member-2-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,635 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#848482290] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,635 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#712585193] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,633 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:50,637 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:50,637 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:41:50,637 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:50,637 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,637 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:50,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:41:50,637 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:50,637 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:50,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:41:50,637 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:41:50,637 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:41:50,637 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:50,637 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:41:50,639 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:50,639 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:50,643 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:50,644 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,644 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,644 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 08:41:50,644 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,644 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:50,644 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:41:50,644 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveAllShardReplicas is initialized 08:41:50,644 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 08:41:50,644 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 08:41:50,644 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:41:50,644 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveAllShardReplicas is initialized 08:41:50,644 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,644 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,644 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 08:41:50,646 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:50,646 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 08:41:50,647 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveAllShardReplicas from null to Follower 08:41:50,647 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:50,657 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-7527545893698554607] 08:41:50,657 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.1.5] ... 08:41:50,662 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:50,662 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:41:50,665 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:50,665 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:50,686 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:50,694 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:50,705 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 08:41:50,712 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#2034099955] 08:41:50,716 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#2034099955] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,717 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:41:50,717 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:41:50,717 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:41:50,717 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveAllShardReplicas 08:41:50,717 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveAllShardReplicas 08:41:50,727 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:41:50,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:41:50,729 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@6aba4759} 08:41:50,729 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 08:41:50,729 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:50,729 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveAllShardReplicas 08:41:50,729 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:50,731 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:50,731 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:41:50,733 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 08:41:50,734 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveAllShardReplicas 08:41:50,734 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-511500976] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,734 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:50,735 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#1140222967 created and ready for shard:member-3-shard-cars-config_testRemoveAllShardReplicas 08:41:50,735 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#1140222967] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,736 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:50,737 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:50,737 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:50,737 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:50,737 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:41:50,738 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-1700309645] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,738 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:50,738 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#-1619864976 created and ready for shard:member-3-shard-people-config_testRemoveAllShardReplicas 08:41:50,739 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#-1619864976] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,739 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:50,739 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1075733247] 08:41:50,739 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1075733247] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,740 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:41:50,740 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:41:50,740 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:41:50,740 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,740 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveAllShardReplicas 08:41:50,741 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:50,741 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:50,742 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:50,742 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,742 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:50,743 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:50,743 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:41:50,743 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:41:50,744 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:50,744 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:41:50,745 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveAllShardReplicas is initialized 08:41:50,745 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,745 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,745 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:50,745 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 08:41:50,745 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:41:50,745 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveAllShardReplicas from null to Follower 08:41:50,746 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:50,747 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:41:50,747 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveAllShardReplicas is initialized 08:41:50,747 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,747 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,747 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 08:41:50,748 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:41:50,748 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 08:41:50,748 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:41:50,748 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveAllShardReplicas from null to Follower 08:41:50,748 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@4b87d44} 08:41:50,749 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 08:41:50,749 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:41:50,749 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:50,749 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,750 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,750 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,750 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-3-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#208831370] 08:41:50,750 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveAllShardReplicas 08:41:50,751 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:50,751 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:50,751 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:41:50,751 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#208831370] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,751 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-3-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#1280421310] 08:41:50,751 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#1280421310] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,751 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-732893625] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,751 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-732893625 created and ready for shard:member-3-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,751 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:50,752 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActor - member-3-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:50,752 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:50,753 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:50,753 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#-442653829] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:50,754 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#-442653829 created and ready for shard:member-3-shard-people-oper_testRemoveAllShardReplicas 08:41:50,754 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:50,755 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:50,756 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:50,758 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:50,759 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:41:50,759 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:50,759 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,759 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveAllShardReplicas is initialized 08:41:50,759 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,759 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 08:41:50,759 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:50,760 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,760 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,760 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 08:41:50,760 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:50,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:41:50,760 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is initialized 08:41:50,760 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 08:41:50,760 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 08:41:50,760 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 08:41:50,760 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveAllShardReplicas from null to Follower 08:41:50,844 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 08:41:50,844 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:50,844 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,844 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,844 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 08:41:50,844 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Follower to Candidate 08:41:50,873 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 08:41:50,873 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:50,874 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,874 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,874 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 08:41:50,874 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Follower to Candidate 08:41:50,915 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 08:41:50,915 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:50,915 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,915 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,915 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 08:41:50,915 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Follower to Candidate 08:41:50,943 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 08:41:50,943 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:50,943 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:50,943 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:50,943 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 08:41:50,943 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Follower to Candidate 08:41:51,193 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 08:41:51,202 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 08:41:51,283 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 08:41:51,293 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 08:41:51,452 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:41:51,452 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:41:51,453 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:51,453 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,453 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,453 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:41:51,453 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:51,453 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:51,453 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,453 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,453 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:41:51,454 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:51,454 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:51,454 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:41:51,454 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,454 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:51,454 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:41:51,454 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:41:51,454 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:41:51,454 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:41:51,454 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:41:51,454 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,454 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:41:51,454 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:51,456 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:41:51,456 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:51,456 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:41:51,456 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:41:51,456 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:41:51,456 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:51,462 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:41:51,462 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:51,462 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:41:51,463 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:41:51,463 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:41:51,463 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:41:51,463 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:41:51,463 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:41:51,573 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:51,573 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:41:51,573 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:41:51,573 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:41:51,573 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:41:51,573 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:51,574 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:51,574 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:51,574 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,574 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:41:51,583 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 08:41:51,585 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:51,585 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:51,593 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:51,593 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 08:41:51,593 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: became leader with frontend state for [] 08:41:51,593 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@797a7458 08:41:51,594 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 08:41:51,594 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Candidate to Leader 08:41:51,594 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveAllShardReplicas status sync done false 08:41:51,594 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@41baf3b7 08:41:51,595 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5bebd3e2 08:41:51,595 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done false 08:41:51,613 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 08:41:51,615 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:51,615 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:51,623 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 08:41:51,623 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:51,623 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 08:41:51,624 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@d82db98 08:41:51,624 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:51,624 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done false 08:41:51,624 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@377a12a 08:41:51,624 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 08:41:51,625 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:51,625 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveAllShardReplicas status sync done false 08:41:51,625 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4f437fc3 08:41:51,625 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 08:41:51,626 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: became leader with frontend state for [] 08:41:51,626 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 08:41:51,626 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Candidate to Leader 08:41:51,626 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 08:41:51,626 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} is now ready 08:41:51,626 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#667600251] 08:41:51,626 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:51,626 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1325220282] 08:41:51,626 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1325220282] 08:41:51,627 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1228210354] 08:41:51,627 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:51,627 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#2014413535] 08:41:51,627 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#2014413535] 08:41:51,628 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#2034099955] 08:41:51,628 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:51,628 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-511500976] 08:41:51,628 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-511500976] 08:41:51,631 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:51,631 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 08:41:51,631 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 08:41:51,631 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 08:41:51,631 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5036b4d9 08:41:51,631 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Candidate to Leader 08:41:51,632 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1441e3e 08:41:51,632 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done false 08:41:51,632 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@520da6d7 08:41:51,633 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveAllShardReplicas status sync done false 08:41:51,639 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@5dc85ac3] 08:41:51,640 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 08:41:51,641 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:51,642 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas-notifier#1654492143 created and ready for shard:member-1-shard-pets-config_testRemoveAllShardReplicas 08:41:51,643 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:51,645 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:51,647 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:51,650 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:51,651 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 08:41:51,651 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testRemoveAllShardReplicas is initialized 08:41:51,651 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:51,651 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:51,651 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 08:41:51,651 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:41:51,651 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from null to Follower 08:41:51,652 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@44469072] 08:41:51,653 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 08:41:51,654 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:51,655 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-2-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:51,655 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas-notifier#-832331829 created and ready for shard:member-2-shard-pets-config_testRemoveAllShardReplicas 08:41:51,657 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:51,660 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:51,662 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:51,663 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 08:41:51,663 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testRemoveAllShardReplicas is initialized 08:41:51,663 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:51,663 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:51,663 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 08:41:51,663 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:41:51,663 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testRemoveAllShardReplicas from null to Follower 08:41:51,665 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@1c85fc81] 08:41:51,666 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 doCreateShard: shardId: member-3-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 08:41:51,667 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#667600251] 08:41:51,667 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 08:41:51,667 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#-752794023] 08:41:51,667 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:41:51,668 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#-752794023] 08:41:51,668 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1228210354] 08:41:51,668 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 08:41:51,668 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-763429570] 08:41:51,668 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-763429570] 08:41:51,668 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#2034099955] 08:41:51,668 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas-notifier#-407448697 created and ready for shard:member-3-shard-pets-config_testRemoveAllShardReplicas 08:41:51,668 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 08:41:51,668 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:41:51,670 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:41:51,671 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:51,669 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 08:41:51,674 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:41:51,674 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [pets] 08:41:51,674 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized 08:41:51,674 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized - notifying 1 OnShardInitialized callbacks 08:41:51,674 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:51,674 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:51,674 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 08:41:51,674 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#22695543] 08:41:51,675 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#22695543] 08:41:51,675 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:41:51,675 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-pets-config_testRemoveAllShardReplicas from null to Follower 08:41:51,677 AM [main] [INFO] ClusterAdminRpcService - Removing replicas for all shards 08:41:51,677 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testRemoveAllShardReplicas 08:41:51,677 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testRemoveAllShardReplicas 08:41:51,677 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 08:41:51,677 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 08:41:51,678 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 08:41:51,678 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 08:41:51,678 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:51,678 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, primaryVersion=13] 08:41:51,678 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 08:41:51,678 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=pets, memberName=MemberName{name=member-3}] 08:41:51,678 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-people-oper_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 08:41:51,678 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:51,678 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, primaryVersion=13] 08:41:51,678 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 08:41:51,678 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas for shard member-3-shard-cars-oper_testRemoveAllShardReplicas 08:41:51,678 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, primaryVersion=13] 08:41:51,679 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 08:41:51,679 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 08:41:51,679 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas for shard member-3-shard-cars-config_testRemoveAllShardReplicas 08:41:51,680 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas for shard member-3-shard-people-config_testRemoveAllShardReplicas 08:41:51,684 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:41:51,686 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveAllShardReplicas} for shard cars 08:41:51,686 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testRemoveAllShardReplicas} for shard cars 08:41:51,686 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 08:41:51,686 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard cars 08:41:51,686 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-config_testRemoveAllShardReplicas} for shard people 08:41:51,686 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard people 08:41:51,686 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:41:51,686 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:41:51,686 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 1 08:41:51,686 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:41:51,686 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 2 08:41:51,686 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 3 08:41:51,687 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#208831370] with 27000 ms timeout 08:41:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard cars has been removed 08:41:51,688 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [people] 08:41:51,689 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 08:41:51,689 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveAllShardReplicas: journal closed 08:41:51,689 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#208831370] 08:41:51,692 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:51,692 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:51,693 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done true 08:41:51,693 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done true 08:41:51,693 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done true 08:41:51,693 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-511500976] with 27000 ms timeout 08:41:51,693 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 08:41:51,693 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets, people] 08:41:51,694 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 08:41:51,694 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveAllShardReplicas: journal closed 08:41:51,695 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-511500976] 08:41:51,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-1700309645] with 27000 ms timeout 08:41:51,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard people has been removed 08:41:51,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets] 08:41:51,696 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 08:41:51,696 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveAllShardReplicas: journal closed 08:41:51,697 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-1700309645] 08:41:51,713 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 08:41:51,715 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:51,715 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:51,722 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:41:51,722 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 08:41:51,722 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@50d8fb8 08:41:51,723 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 08:41:51,723 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 08:41:51,723 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Candidate to Leader 08:41:51,723 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 08:41:51,723 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done false 08:41:51,723 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveAllShardReplicas status sync done false 08:41:51,723 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@64e20de8 08:41:51,724 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 08:41:51,724 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@75ebd688 08:41:51,724 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 08:41:51,724 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, primaryVersion=13] 08:41:51,724 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 08:41:51,724 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas for shard member-3-shard-people-oper_testRemoveAllShardReplicas 08:41:51,727 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-oper_testRemoveAllShardReplicas} for shard people 08:41:51,727 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard people 08:41:51,727 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:41:51,727 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 4 08:41:51,731 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:51,731 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done true 08:41:51,732 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#1280421310] with 27000 ms timeout 08:41:51,732 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard people has been removed 08:41:51,732 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [] 08:41:51,732 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 08:41:51,733 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveAllShardReplicas: journal closed 08:41:51,733 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#1280421310] 08:41:52,14 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 08:41:52,14 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:41:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:52,14 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 08:41:52,15 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Follower to Candidate 08:41:52,16 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:52,17 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:41:52,25 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:52,25 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 08:41:52,25 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: became leader with frontend state for [] 08:41:52,25 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 08:41:52,25 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@33c1c8 08:41:52,25 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Candidate to Leader 08:41:52,25 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 08:41:52,26 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2368dd 08:41:52,26 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@266fdaa2 08:41:52,27 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 08:41:52,27 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, primaryVersion=13] 08:41:52,27 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 08:41:52,27 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 08:41:52,27 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-pets-config_testRemoveAllShardReplicas status sync done false 08:41:52,27 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done false 08:41:52,27 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas for shard member-3-shard-pets-config_testRemoveAllShardReplicas 08:41:52,30 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-pets-config_testRemoveAllShardReplicas} for shard pets 08:41:52,31 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard pets 08:41:52,31 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=pets}, type Config 08:41:52,31 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 5 08:41:52,33 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#667600251] 08:41:52,33 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:52,33 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1325220282] 08:41:52,33 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1325220282] 08:41:52,34 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#667600251] 08:41:52,34 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:41:52,34 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-1395913076] 08:41:52,34 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-1395913076] 08:41:52,34 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#667600251] 08:41:52,34 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 08:41:52,34 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#-752794023] 08:41:52,34 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#-752794023] 08:41:52,35 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorVotingConfigSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:52,35 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1228210354] 08:41:52,35 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:52,35 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#2014413535] 08:41:52,35 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#2014413535] 08:41:52,35 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1228210354] 08:41:52,35 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done true 08:41:52,35 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:41:52,36 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#1247411242] 08:41:52,36 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#1247411242] 08:41:52,36 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#22695543] with 27000 ms timeout 08:41:52,36 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard pets has been removed 08:41:52,36 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [] 08:41:52,36 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1228210354] 08:41:52,36 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 08:41:52,37 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 08:41:52,37 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-763429570] 08:41:52,37 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-763429570] 08:41:52,37 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-pets-config_testRemoveAllShardReplicas: journal closed 08:41:52,37 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#2034099955] 08:41:52,37 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#22695543] 08:41:52,38 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:52,38 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [] 08:41:52,38 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#2034099955] 08:41:52,38 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:41:52,38 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard people not found - shards present: [] 08:41:52,39 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#2034099955] 08:41:52,39 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 08:41:52,39 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard pets not found - shards present: [] 08:41:52,39 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:41:52,39 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 0 shards 08:41:52,40 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:41:52,40 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:41:52,40 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 08:41:52,40 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 0 shards 08:41:52,40 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:41:52,41 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:41:52,42 AM [cluster-test-shard-dispatcher-18] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveAllShardReplicas (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 314, lastApplied : 0, commitIndex : 0 08:41:52,43 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:52,43 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:52,48 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:52,48 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:52,48 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:52,48 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:52,50 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:41:52,56 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:41:52,56 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testRemoveAllShardReplicas 08:41:52,56 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveAllShardReplicas 08:41:52,56 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveAllShardReplicas 08:41:52,56 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 08:41:52,57 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 08:41:52,57 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 08:41:52,57 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 08:41:52,57 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveAllShardReplicas: journal closed 08:41:52,57 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveAllShardReplicas: journal closed 08:41:52,57 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testRemoveAllShardReplicas: journal closed 08:41:52,58 AM [cluster-test-pekko.actor.default-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:41:52,58 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:41:52,58 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 08:41:52,58 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveAllShardReplicas 08:41:52,58 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveAllShardReplicas 08:41:52,58 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:41:52,59 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 08:41:52,59 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 08:41:52,59 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveAllShardReplicas: journal closed 08:41:52,60 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveAllShardReplicas: journal closed 08:41:52,60 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:41:52,60 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:41:52,61 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:52,61 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:52,470 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:41:53,61 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-7527545893698554607] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:41:53,79 AM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:53,80 AM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:53,80 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [4867104681791588656] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:41:53,81 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:41:53.085] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:53,95 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:41:53,95 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testRemoveAllShardReplicas 08:41:53,96 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveAllShardReplicas 08:41:53,96 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveAllShardReplicas 08:41:53,96 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 08:41:53,96 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@11281747 08:41:53,96 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5820554d 08:41:53,96 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4af77b52 08:41:53,96 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5820554d 08:41:53,96 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 08:41:53,96 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-pets-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4af77b52 08:41:53,96 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 08:41:53,96 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@f6dd0e1 08:41:53,96 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@95c8be6 08:41:53,96 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@95c8be6 08:41:53,96 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 08:41:53,97 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1d8dbf38 08:41:53,511 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer aborted 08:41:53,511 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: Failed to transfer leadership in 414.6 ms 08:41:53,511 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In unpauseLeader 08:41:53,512 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 08:41:53,512 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveAllShardReplicas: journal closed 08:41:53,540 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer aborted 08:41:53,540 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: Failed to transfer leadership in 444.1 ms 08:41:53,540 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In unpauseLeader 08:41:53,541 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 08:41:53,541 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveAllShardReplicas: journal closed 08:41:53,620 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer aborted 08:41:53,620 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: Failed to transfer leadership in 524.0 ms 08:41:53,620 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In unpauseLeader 08:41:53,620 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 08:41:53,621 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testRemoveAllShardReplicas: journal closed 08:41:53,622 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:41:53,622 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:41:53,622 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 08:41:53,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveAllShardReplicas 08:41:53,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveAllShardReplicas 08:41:53,623 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:41:53,623 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4c4e11e1 08:41:53,623 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@23ee5dfb 08:41:53,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@23ee5dfb 08:41:53,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 08:41:53,624 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2d453a51 08:41:53,624 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1e8d5854 08:41:53,624 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1e8d5854 08:41:53,624 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 08:41:53,960 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer aborted 08:41:53,960 AM [cluster-test-shard-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 336.9 ms 08:41:53,961 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In unpauseLeader 08:41:53,961 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 08:41:53,962 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveAllShardReplicas: journal closed 08:41:54,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer aborted 08:41:54,191 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 567.3 ms 08:41:54,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In unpauseLeader 08:41:54,191 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 08:41:54,192 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveAllShardReplicas: journal closed 08:41:54,192 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:41:54,192 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:41:54,193 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:54,194 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:54,197 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:41:54.200] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:54,224 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:41:54,234 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-1381726532721981407] 08:41:54,234 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:41:54,238 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:41:54,238 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:41:54,238 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:54,238 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:54,250 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:54,251 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:41:54,251 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:54,251 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:41:54,251 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:41:54,253 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplicaFailures 08:41:54,255 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1803587457] 08:41:54,255 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:41:54,255 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:41:54,255 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:41:54,255 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplicaFailures 08:41:54,257 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1803587457] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,258 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:54,258 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 08:41:54,275 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 08:41:54,276 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@72b54f3d} 08:41:54,277 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} started 08:41:54,277 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:54,279 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:54,283 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplicaFailures 08:41:54,283 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:54,283 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplicaFailures 08:41:54,284 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 08:41:54,284 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 08:41:54,284 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures on shard member-1-shard-cars-config_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#944860737] 08:41:54,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#2137844276] 08:41:54,285 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#2137844276] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,285 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:41:54,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:41:54,285 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:41:54,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplicaFailures 08:41:54,286 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:54,286 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 08:41:54,286 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#944860737] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,286 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Shard created, persistent : true 08:41:54,287 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#-909670977 created and ready for shard:member-1-shard-cars-config_testAddShardReplicaFailures 08:41:54,287 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#-909670977] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,287 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-cars-config_testAddShardReplicaFailures: Starting recovery with journal batch size 1 08:41:54,289 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplicaFailures: journal open: applyTo=0 08:41:54,290 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:54,290 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 08:41:54,291 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@7c72af1b} 08:41:54,291 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} started 08:41:54,291 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:54,291 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplicaFailures 08:41:54,294 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=people} 08:41:54,294 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 08:41:54,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 08:41:54,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures on shard member-1-shard-cars-oper_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-2109500960] 08:41:54,295 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Shard created, persistent : true 08:41:54,295 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-2109500960] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,295 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: onAddShardReplica: AddShardReplica[ShardName=people] 08:41:54,296 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#1550124127 created and ready for shard:member-1-shard-cars-oper_testAddShardReplicaFailures 08:41:54,296 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: No module configuration exists for shard people 08:41:54,296 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testAddShardReplicaFailures: Starting recovery with journal batch size 1 08:41:54,296 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#1550124127] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,297 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplicaFailures: journal open: applyTo=0 08:41:54,298 AM [cluster-test-pekko.actor.default-dispatcher-6] [ERROR] ClusterAdminRpcService - Failed to add replica for shard ShardName{value=people} java.lang.IllegalArgumentException: No module configuration exists for shard people at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onAddShardReplica(ShardManager.java:1138) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleReceive(ShardManager.java:260) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:272) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:233) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:245) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 08:41:54,298 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:54,299 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 08:41:54,299 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplicaFailures 08:41:54,299 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 08:41:54,301 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplicaFailures: Local TermInfo store seeded with TermInfo{term=0} 08:41:54,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 08:41:54,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:54,302 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from null to Follower 08:41:54,302 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:41:54,302 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplicaFailures is initialized 08:41:54,302 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplicaFailures: Local TermInfo store seeded with TermInfo{term=0} 08:41:54,303 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:41:54,303 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplicaFailures is initialized 08:41:54,303 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 08:41:54,304 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Starting new election term 1 08:41:54,305 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:54,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 08:41:54,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:54,305 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Follower to Candidate 08:41:54,305 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:54,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 08:41:54,305 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: became leader with frontend state for [] 08:41:54,305 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Candidate to Leader 08:41:54,305 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:54,305 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures 08:41:54,305 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from null to Follower 08:41:54,305 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2b3fc6af 08:41:54,305 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplicaFailures from Candidate to Leader 08:41:54,305 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplicaFailures is ready 08:41:54,307 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Starting new election term 1 08:41:54,307 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:54,307 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 08:41:54,307 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:54,307 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Follower to Candidate 08:41:54,307 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:54,307 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 08:41:54,307 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: became leader with frontend state for [] 08:41:54,307 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Candidate to Leader 08:41:54,307 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures 08:41:54,307 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@519f7ae9 08:41:54,308 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Stopping Shard 08:41:54,308 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplicaFailures from Candidate to Leader 08:41:54,308 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplicaFailures is ready 08:41:54,308 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplicaFailures: journal closed 08:41:54,308 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:41:54,308 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:41:54,309 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 08:41:54,309 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1803587457] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1803587457] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,309 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1803587457] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-1803587457] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,309 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f0d54803ba0] to Actor[pekko://cluster-test/user/$a#1185449560] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#1185449560] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,309 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplicaFailures 08:41:54,309 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 08:41:54,310 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@110254d 08:41:54,310 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@110254d 08:41:54,310 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Stopping Shard 08:41:54,310 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplicaFailures: journal closed 08:41:54,311 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:41:54,311 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:41:54,321 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#2137844276] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#2137844276] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,321 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:54,321 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:54,323 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:41:54,346 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:41:54,404 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-6749666332652908133] 08:41:54,405 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:41:54,411 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:41:54,411 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:41:54,412 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:54,412 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:54,427 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:41:54,428 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:41:54,428 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:41:54,429 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:54,431 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:54,434 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 08:41:54,436 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1056206697] 08:41:54,437 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:41:54,437 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:41:54,437 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:41:54,437 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplica 08:41:54,438 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:54,438 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:41:54,439 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1056206697] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,447 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 08:41:54,447 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@773a6270} 08:41:54,448 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} started 08:41:54,449 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:54,449 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplica 08:41:54,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:41:54,450 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:41:54,450 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica on shard member-1-shard-cars-config_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478] 08:41:54,450 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,450 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Shard created, persistent : true 08:41:54,456 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#212373323 created and ready for shard:member-1-shard-cars-config_testAddShardReplica 08:41:54,456 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 08:41:54,456 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#212373323] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,457 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplica: journal open: applyTo=0 08:41:54,458 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:54,461 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:54,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:54,462 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:41:54,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:54,462 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplica is initialized 08:41:54,462 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from null to Follower 08:41:54,464 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testAddShardReplica (Candidate): Starting new election term 1 08:41:54,464 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:54,464 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:54,464 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:54,464 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Follower to Candidate 08:41:54,465 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:54,465 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:54,465 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: became leader with frontend state for [] 08:41:54,465 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Candidate to Leader 08:41:54,465 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 08:41:54,465 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1484f16d 08:41:54,465 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Candidate to Leader 08:41:54,465 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 08:41:54,499 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1056206697] 08:41:54,499 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:54,499 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478] 08:41:54,500 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:54,502 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:54,506 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 08:41:54,508 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#526982037] 08:41:54,509 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#526982037] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,510 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:41:54,510 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:41:54,510 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:41:54,510 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplica 08:41:54,511 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:54,511 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 08:41:54,511 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 08:41:54,512 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@11226c89} 08:41:54,512 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} started 08:41:54,513 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:54,513 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplica 08:41:54,513 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,514 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 08:41:54,514 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 08:41:54,514 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 08:41:54,514 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#1894746301] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,514 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#1894746301 created and ready for shard:member-1-shard-cars-oper_testAddShardReplica 08:41:54,514 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica on shard member-1-shard-cars-oper_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577] 08:41:54,514 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 08:41:54,516 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 08:41:54,517 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:54,519 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:54,519 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:41:54,519 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:54,519 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:54,519 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplica is initialized 08:41:54,519 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from null to Follower 08:41:54,523 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-oper_testAddShardReplica (Candidate): Starting new election term 1 08:41:54,523 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:41:54,523 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:54,523 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:54,523 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Follower to Candidate 08:41:54,523 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:41:54,523 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:41:54,523 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: became leader with frontend state for [] 08:41:54,523 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Candidate to Leader 08:41:54,523 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 08:41:54,523 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@140b8b5b 08:41:54,523 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Candidate to Leader 08:41:54,523 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:54,563 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#526982037] 08:41:54,563 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:41:54,563 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577] 08:41:54,569 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:41:54,579 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [1260362309976744035] 08:41:54,579 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.1.5] ... 08:41:54,581 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:54,582 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:41:54,583 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:54,583 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:54,603 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:54,605 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:54,609 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 08:41:54,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#889165531] 08:41:54,612 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:41:54,612 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:41:54,612 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:41:54,612 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#889165531] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,639 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 08:41:54,640 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@349f7363} 08:41:54,643 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} started 08:41:54,644 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:54,644 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:54,645 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:54,647 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 08:41:54,648 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:41:54,649 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#656760186] 08:41:54,649 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#656760186] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:54,649 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:41:54,650 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:41:54,650 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:41:54,651 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:41:54,655 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 08:41:54,656 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} started 08:41:54,658 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:54,658 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 08:41:54,658 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:54,658 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:54,658 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@3a589b80} 08:41:54,658 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:55,431 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:41:55,431 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:55,432 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 08:41:55,432 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 08:41:55,432 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:55,432 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 08:41:55,432 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:41:55,432 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:55,433 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 08:41:55,458 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:41:55,458 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 08:41:55,459 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:55,459 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica] 08:41:55,462 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:41:55,462 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478, primaryVersion=13] 08:41:55,464 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478 for shard member-2-shard-cars-config_testAddShardReplica 08:41:55,465 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Shard created, persistent : true 08:41:55,465 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1959202774] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:55,466 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 08:41:55,466 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#1967109476 created and ready for shard:member-2-shard-cars-config_testAddShardReplica 08:41:55,467 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#1967109476] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:55,467 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddShardReplica 08:41:55,468 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:41:55,468 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:55,469 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddShardReplica: journal open: applyTo=0 08:41:55,471 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:55,473 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: snapshot is durable as of 2025-08-30T08:41:55.468262130Z 08:41:55,474 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:55,474 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:41:55,474 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddShardReplica is initialized 08:41:55,474 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:55,474 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:55,474 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , received role change from null to Follower 08:41:55,475 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:41:55,477 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-config_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddShardReplica 08:41:55,478 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 08:41:55,478 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-config_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:55,478 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@53d782a4 08:41:55,478 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddShardReplica from null to Follower 08:41:55,478 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 08:41:55,478 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 08:41:55,481 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-config_testAddShardReplica: snapshot is durable as of 2025-08-30T08:41:55.478547878Z 08:41:55,481 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddShardReplica: Applying snapshot 08:41:55,481 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:55,481 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#1089474858] 08:41:55,481 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#1089474858] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:55,481 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#1089474858] with name member-2-shard-cars-config_testAddShardReplica-DTCL-publisher 08:41:55,481 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 08:41:55,482 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 99.51 μs 08:41:55,482 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: state snapshot applied in 1.023 ms 08:41:55,482 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddShardReplica: Done applying snapshot 08:41:55,483 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:41:55,485 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478 08:41:55,485 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 08:41:55,485 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddShardReplica 08:41:55,486 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 08:41:55,487 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:55,490 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5d1076ae 08:41:55,490 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:41:55,490 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 08:41:55,490 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#889165531] 08:41:55,490 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:55,490 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 08:41:55,490 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:55,490 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1959202774] 08:41:55,490 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1959202774] 08:41:55,490 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 08:41:55,490 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#656760186] 08:41:55,490 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:41:55,490 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 08:41:55,490 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:41:55,491 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 08:41:55,491 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:55,491 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica] 08:41:55,492 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:41:55,492 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577, primaryVersion=13] 08:41:55,494 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577 for shard member-2-shard-cars-oper_testAddShardReplica 08:41:55,494 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 08:41:55,494 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#617836477] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:55,495 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#1433819634 created and ready for shard:member-2-shard-cars-oper_testAddShardReplica 08:41:55,495 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#1433819634] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:55,496 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddShardReplica 08:41:55,496 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 08:41:55,496 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:41:55,497 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:55,500 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 08:41:55,500 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2025-08-30T08:41:55.496640941Z 08:41:55,501 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:55,504 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:55,504 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:41:55,504 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddShardReplica is initialized 08:41:55,504 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:55,504 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:55,504 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , received role change from null to Follower 08:41:55,504 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:41:55,506 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-oper_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddShardReplica 08:41:55,506 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 08:41:55,506 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:55,506 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 08:41:55,506 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@592fe9d4 08:41:55,506 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddShardReplica from null to Follower 08:41:55,506 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:55,509 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2025-08-30T08:41:55.507104404Z 08:41:55,509 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot 08:41:55,509 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:55,510 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#2902380] with name member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher 08:41:55,510 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#2902380] 08:41:55,510 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: state snapshot applied in 477.8 μs 08:41:55,510 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddShardReplica: Done applying snapshot 08:41:55,510 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 08:41:55,510 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#2902380] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:55,510 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 148.2 μs 08:41:55,512 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:41:55,514 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577 08:41:55,514 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 08:41:55,514 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddShardReplica 08:41:55,514 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 08:41:55,517 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:55,518 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 08:41:55,518 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:55,519 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1c2d7c95 08:41:55,519 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:41:55,519 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:55,519 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 08:41:55,519 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#656760186] 08:41:55,519 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:41:55,519 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#617836477] 08:41:55,519 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#617836477] 08:41:55,525 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:55,538 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-4389148212793194829] 08:41:55,538 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.1.5] ... 08:41:55,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:41:55,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:41:55,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:55,544 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:55,552 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:55,553 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:55,555 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 08:41:55,561 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1239135926] 08:41:55,562 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:41:55,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:41:55,562 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:41:55,564 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1239135926] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:55,569 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 08:41:55,569 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@521f585d} 08:41:55,569 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} started 08:41:55,570 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:55,571 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:55,571 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:55,573 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 08:41:55,574 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1428678802] 08:41:55,574 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1428678802] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:55,575 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:41:55,575 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:41:55,575 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:41:55,583 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 08:41:55,591 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@5cd8bc6e} 08:41:55,591 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} started 08:41:55,592 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:55,597 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:41:55,599 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:41:55,603 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:55,603 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 08:41:55,603 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 08:41:55,604 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:55,604 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 08:41:55,604 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 08:41:55,604 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:55,605 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:55,605 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:55,605 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:55,605 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:55,605 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:41:55,605 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:55,605 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:41:55,608 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 08:41:55,608 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:56,102 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 08:41:56,131 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 08:41:56,421 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 08:41:56,450 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:41:56,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:56,450 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:56,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 08:41:56,450 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 08:41:56,450 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:41:56,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:41:56,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:56,450 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 08:41:56,452 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 08:41:56,458 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:56,458 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:41:56,458 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 08:41:56,458 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:56,458 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 08:41:56,458 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 08:41:56,458 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 08:41:56,458 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:56,494 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:41:56,494 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 08:41:56,495 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:56,495 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica] 08:41:56,497 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:41:56,497 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478, primaryVersion=13] 08:41:56,499 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478 for shard member-3-shard-cars-config_testAddShardReplica 08:41:56,500 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Shard created, persistent : true 08:41:56,500 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1566687075] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,501 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#592237944 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 08:41:56,501 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 08:41:56,501 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#592237944] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,501 AM [cluster-test-shard-dispatcher-16] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testAddShardReplica 08:41:56,502 AM [cluster-test-shard-dispatcher-16] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 08:41:56,502 AM [cluster-test-shard-dispatcher-16] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:56,503 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal open: applyTo=0 08:41:56,505 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:56,506 AM [cluster-test-shard-dispatcher-16] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: snapshot is durable as of 2025-08-30T08:41:56.502638111Z 08:41:56,508 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:56,508 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:41:56,508 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 08:41:56,508 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:56,508 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:56,509 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 08:41:56,509 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]}}" message is greater than follower's term 0 - updating term 08:41:56,511 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-cars-config_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddShardReplica 08:41:56,511 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 08:41:56,511 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7094207c 08:41:56,511 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-3-shard-cars-config_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:56,511 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 08:41:56,511 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:41:56,511 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 08:41:56,514 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-3-shard-cars-config_testAddShardReplica: snapshot is durable as of 2025-08-30T08:41:56.511971364Z 08:41:56,514 AM [cluster-test-shard-dispatcher-11] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testAddShardReplica: Applying snapshot 08:41:56,514 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:56,514 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1132139355] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 08:41:56,514 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 466.1 μs 08:41:56,514 AM [cluster-test-shard-dispatcher-11] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testAddShardReplica: Done applying snapshot 08:41:56,514 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1132139355] 08:41:56,515 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 08:41:56,515 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#1132139355] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,515 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 159.8 μs 08:41:56,516 AM [cluster-test-shard-dispatcher-16] [INFO] AbstractLeader - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 08:41:56,518 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7e53c01e 08:41:56,518 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:41:56,518 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478 08:41:56,519 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 08:41:56,519 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testAddShardReplica 08:41:56,519 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 08:41:56,519 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:56,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 08:41:56,523 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:56,523 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 08:41:56,523 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:41:56,523 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1239135926] 08:41:56,523 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:56,523 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1566687075] 08:41:56,523 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1566687075] 08:41:56,523 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1428678802] 08:41:56,523 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:41:56,523 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 08:41:56,524 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:41:56,524 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 08:41:56,524 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:56,524 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica] 08:41:56,525 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:41:56,525 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577, primaryVersion=13] 08:41:56,527 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577 for shard member-3-shard-cars-oper_testAddShardReplica 08:41:56,528 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 08:41:56,528 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#921097349] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,528 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#-1687740898 created and ready for shard:member-3-shard-cars-oper_testAddShardReplica 08:41:56,528 AM [cluster-test-shard-dispatcher-16] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testAddShardReplica 08:41:56,528 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 08:41:56,528 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#-1687740898] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,529 AM [cluster-test-shard-dispatcher-16] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 08:41:56,529 AM [cluster-test-shard-dispatcher-16] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:41:56,529 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 08:41:56,531 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:41:56,532 AM [cluster-test-shard-dispatcher-16] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2025-08-30T08:41:56.529253845Z 08:41:56,533 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:41:56,533 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:41:56,533 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testAddShardReplica is initialized 08:41:56,533 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:56,533 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:56,533 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , received role change from null to Follower 08:41:56,533 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]}}" message is greater than follower's term 0 - updating term 08:41:56,536 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-cars-oper_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddShardReplica 08:41:56,536 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:41:56,536 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 08:41:56,536 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 08:41:56,536 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7a6c4b0c 08:41:56,536 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testAddShardReplica from null to Follower 08:41:56,536 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:56,538 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-3-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2025-08-30T08:41:56.536493354Z 08:41:56,538 AM [cluster-test-shard-dispatcher-11] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot 08:41:56,538 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:56,539 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#1885367587] with name member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher 08:41:56,539 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: state snapshot applied in 368.9 μs 08:41:56,539 AM [cluster-test-shard-dispatcher-11] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testAddShardReplica: Done applying snapshot 08:41:56,539 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#1885367587] 08:41:56,539 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 08:41:56,539 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#1885367587] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,539 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 95.97 μs 08:41:56,540 AM [cluster-test-shard-dispatcher-16] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 08:41:56,542 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577 08:41:56,542 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 08:41:56,542 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testAddShardReplica 08:41:56,542 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 08:41:56,543 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:41:56,546 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3f6eca25 08:41:56,546 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:56,546 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:41:56,546 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1428678802] 08:41:56,546 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:41:56,546 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#921097349] 08:41:56,546 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#921097349] 08:41:56,546 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#889165531] 08:41:56,546 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 08:41:56,547 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 08:41:56,547 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:56,547 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:56,547 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1959202774] 08:41:56,547 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1959202774] 08:41:56,547 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#656760186] 08:41:56,547 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:41:56,547 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#617836477] 08:41:56,547 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#617836477] 08:41:56,559 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:56,559 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 08:41:56,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 08:41:56,559 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 08:41:56,559 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:56,559 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:41:56,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:41:56,560 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:41:56,595 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#889165531] 08:41:56,595 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:56,595 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 08:41:56,595 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 08:41:56,608 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 08:41:56,621 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=72052, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 08:41:56,628 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 08:41:56,630 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 08:41:56,654 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 24.77 ms 08:41:56,657 AM [cluster-test-shard-dispatcher-16] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,681 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: readying transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,681 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 ready 08:41:56,686 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: transitioned member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 to ready 08:41:56,686 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 08:41:56,690 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Validating transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,690 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 validated 08:41:56,690 AM [cluster-test-shard-dispatcher-16] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 08:41:56,691 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 08:41:56,691 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Preparing transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,695 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 candidate ready 08:41:56,697 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 08:41:56,701 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 08:41:56,701 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 08:41:56,701 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 prepared 08:41:56,701 AM [cluster-test-shard-dispatcher-16] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 08:41:56,701 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 08:41:56,702 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Starting commit for transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,708 AM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 08:41:56,710 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 08:41:56,715 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 08:41:56,716 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Resuming commit of transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,718 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendMetadata - member-1-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@56250039}, purged=MutableUnsignedLongSet{size=0}} 08:41:56,718 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,719 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,720 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 08:41:56,720 AM [cluster-test-shard-dispatcher-16] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 08:41:56,722 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#497118449] with name member-1-shard-cars-config_testAddShardReplica-DTCL-publisher 08:41:56,722 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#497118449] 08:41:56,722 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 08:41:56,722 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-config_testAddShardReplica-DTCL-publisher: 113.7 μs 08:41:56,722 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#497118449] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,732 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1239135926] 08:41:56,732 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:56,732 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 08:41:56,732 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 08:41:56,735 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,736 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,737 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,738 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@481deb03}, purged=MutableUnsignedLongSet{size=0}} 08:41:56,739 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,739 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,739 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 08:41:56,739 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5082acd5}, purged=MutableUnsignedLongSet{size=0}} 08:41:56,739 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,739 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,739 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 183.8 μs 08:41:56,739 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 08:41:56,739 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 124.4 μs 08:41:56,740 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 08:41:56,744 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 08:41:56,744 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=14140, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 08:41:56,749 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 08:41:56,749 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 08:41:56,750 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 724.7 μs 08:41:56,750 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: replication of member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f0d5496efa8@526b0643 08:41:56,751 AM [cluster-test-shard-dispatcher-16] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: finished purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,752 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,752 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,755 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,755 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,756 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,756 AM [cluster-test-shard-dispatcher-16] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,756 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,763 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1428678802] 08:41:56,764 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:56,764 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 08:41:56,764 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 08:41:56,769 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 08:41:56,769 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=17070, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 08:41:56,770 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 08:41:56,770 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 08:41:56,771 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 967.7 μs 08:41:56,772 AM [cluster-test-shard-dispatcher-16] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: readying transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 ready 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: transitioned member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 to ready 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Validating transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 validated 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Preparing transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 candidate ready 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 prepared 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Starting commit for transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,773 AM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 233 08:41:56,774 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 08:41:56,776 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Resuming commit of transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,776 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 08:41:56,776 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@591a1ad}, purged=MutableUnsignedLongSet{size=0}} 08:41:56,776 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,777 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,777 AM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 08:41:56,777 AM [cluster-test-shard-dispatcher-16] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 08:41:56,777 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-859597510] with name member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher 08:41:56,777 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-859597510] 08:41:56,777 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 08:41:56,777 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher: 133.5 μs 08:41:56,777 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-859597510] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,779 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,779 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,779 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@22f2cf32}, purged=MutableUnsignedLongSet{size=0}} 08:41:56,779 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,779 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,779 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 08:41:56,779 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 123.5 μs 08:41:56,780 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#656760186] 08:41:56,781 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:56,781 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 08:41:56,781 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,781 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 08:41:56,781 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@617eae50}, purged=MutableUnsignedLongSet{size=0}} 08:41:56,781 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,781 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,781 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 08:41:56,782 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 08:41:56,782 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 108.1 μs 08:41:56,782 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 08:41:56,784 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=17550, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 08:41:56,784 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f0d5496efa8@f08dfb9 08:41:56,784 AM [cluster-test-shard-dispatcher-16] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,784 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,784 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,785 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 08:41:56,785 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 08:41:56,785 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 339.5 μs 08:41:56,786 AM [cluster-test-shard-dispatcher-16] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open snapshot member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,787 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,787 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,788 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,788 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,788 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1056206697] 08:41:56,788 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:41:56,789 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478] 08:41:56,789 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478] 08:41:56,789 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#526982037] 08:41:56,789 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:41:56,789 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577] 08:41:56,789 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577] 08:41:56,789 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#889165531] 08:41:56,789 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:41:56,789 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1959202774] 08:41:56,789 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-1959202774] 08:41:56,789 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#656760186] 08:41:56,790 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:41:56,790 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#617836477] 08:41:56,790 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#617836477] 08:41:56,790 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1239135926] 08:41:56,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:56,790 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1566687075] 08:41:56,790 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1566687075] 08:41:56,790 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1428678802] 08:41:56,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:41:56,790 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#921097349] 08:41:56,790 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#921097349] 08:41:56,790 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 08:41:56,790 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 08:41:56,791 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 08:41:56,791 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2557] as [Down] 08:41:56,791 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Stopping Shard 08:41:56,792 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal closed 08:41:56,792 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:41:56,793 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:41:56,793 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 08:41:56,793 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testAddShardReplica 08:41:56,793 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 08:41:56,794 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 08:41:56,794 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Node has been marked as DOWN. Shutting down myself 08:41:56,794 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f0d54803ba0] to Actor[pekko://cluster-test/user/$a#566419888] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#566419888] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,794 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Shutting down... 08:41:56,794 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Successfully shut down 08:41:56,794 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Stopping Shard 08:41:56,795 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1239135926] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1239135926] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,795 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:41:56,795 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:41:56,795 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:41:56,796 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testAddShardReplica: journal closed 08:41:56,797 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:41:56,797 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:41:56,801 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:41:56,801 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:41:56,806 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [08/30/2025 08:41:56.805] [cluster-test-pekko.remote.default-remote-dispatcher-6] [org.apache.pekko.stream.Log(pekko://cluster-test/system/Materializers/StreamSupervisor-58)] [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [08/30/2025 08:41:56.805] [cluster-test-pekko.remote.default-remote-dispatcher-6] [org.apache.pekko.stream.Log(pekko://cluster-test/system/Materializers/StreamSupervisor-58)] [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [08/30/2025 08:41:56.805] [cluster-test-pekko.remote.default-remote-dispatcher-9] [org.apache.pekko.stream.Log(pekko://cluster-test/system/Materializers/StreamSupervisor-58)] [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:41:56,839 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:41:56,853 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-3267786605008534065] 08:41:56,853 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.1.5] ... 08:41:56,857 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Registered cluster JMX MBean [pekko:type=Cluster] 08:41:56,857 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:41:56,861 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:41:56,862 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:41:56,873 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-0-txn-0-0} 08:41:56,873 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-0} 08:41:56,877 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: aborting transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:56,878 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:56,879 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:41:56,881 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 08:41:56,887 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#389906779] 08:41:56,888 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:41:56,888 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: applying snapshot ShardManagerSnapshot [ShardList = [cars] ] 08:41:56,888 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: adding shard cars 08:41:56,888 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:41:56,888 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testAddShardReplica 08:41:56,889 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#389906779] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,902 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@58d15c88} 08:41:56,902 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} started 08:41:56,902 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#389906779] 08:41:56,903 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:41:56,903 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testAddShardReplica 08:41:56,907 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:41:56,908 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 08:41:56,908 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Shard created, persistent : true 08:41:56,909 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 08:41:56,912 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-48901178] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,912 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#1741317048 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 08:41:56,912 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#1741317048] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,915 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal open: applyTo=3 08:41:56,917 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: Applying recovered snapshot 08:41:56,923 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:41:56,925 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-552209324] 08:41:56,925 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-552209324] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 08:41:56,925 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 08:41:56,925 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-552209324] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:56,925 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 2.396 ms 08:41:56,925 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 94.29 μs 08:41:56,925 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - Recovery snapshot applied for member-3-shard-cars-config_testAddShardReplica in 7.895 ms: snapshotIndex=0, snapshotTerm=1, journal-size=0 08:41:56,925 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testAddShardReplica: Recovery completed in 8.057 ms - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = 0, snapshot term = 1, journal size = 0 08:41:56,926 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 08:41:56,932 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@cb39cde}, purged=MutableUnsignedLongSet{size=0}} 08:41:56,932 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,933 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,933 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 08:41:56,933 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:56,933 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:41:56,933 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:41:56,933 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:41:56,933 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 08:41:56,934 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:41:56,934 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 08:41:56,934 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 08:41:56,934 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-48901178] 08:41:56,934 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-48901178] 08:41:56,935 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 08:41:56,936 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#389906779] 08:41:56,936 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:41:56,936 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 08:41:56,936 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 08:41:56,936 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 08:41:57,93 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda/0x00007f0d54975968@29df6400 08:41:57,93 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@2bf7069f}, purged=MutableUnsignedLongSet{size=0}} 08:41:57,94 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:57,95 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Aborted transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:57,98 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@504170c8}, purged=MutableUnsignedLongSet{size=0}} 08:41:57,99 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:57,99 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Aborted transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:57,100 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:57,102 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f0d5496efa8@5dbfb47f 08:41:57,102 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:57,102 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:57,103 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:57,410 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:41:57,411 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:41:57,817 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-4389148212793194829] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:41:57,819 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-4389148212793194829] but current is [-3267786605008534065]. Reason: ActorSystem terminated 08:41:57,876 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] LocalActorRefProvider$$anon$2 - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] wrapped in [org.apache.pekko.actor.ActorSelectionMessage] to Actor[pekko://cluster-test/] was dropped. Unknown system with UID [-6749666332652908133]. This system with UID [-3267786605008534065] was probably restarted. Messages will be accepted when new handshake has been completed.. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:57,877 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:57,877 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:57,878 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@38160092 08:41:57,878 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 08:41:57,878 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:57,879 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 08:41:57,879 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 08:41:57,879 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:41:57,881 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:41:58,52 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:58,340 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:41:58,372 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:41:58,461 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#2004759110]] intended for system uid [-4389148212793194829], self uid is [-3267786605008534065] 08:41:58,901 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:41:59,921 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:42:00,481 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#2004759110]] intended for system uid [-4389148212793194829], self uid is [-3267786605008534065] 08:42:00,658 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#1681539338]] intended for system uid [-4389148212793194829], self uid is [-3267786605008534065] 08:42:00,941 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:42:01,961 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:42:02,502 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#2004759110]] intended for system uid [-4389148212793194829], self uid is [-3267786605008534065] 08:42:02,570 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Down)]. 08:42:02,570 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:02,570 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:02,737 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Down)]. 08:42:02,738 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:02,738 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:02,982 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:42:03,591 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2557] 08:42:03,591 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:03,591 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:03,592 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-4389148212793194829] but current is [-3267786605008534065]. Reason: Cluster member removed, previous status [Down] 08:42:03,762 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:03,762 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:03,762 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-4389148212793194829] but current is [-3267786605008534065]. Reason: Cluster member removed, previous status [Down] 08:42:04,2 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:42:04,5 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:42:04,6 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:04,6 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:42:04,6 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:42:04,6 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is ready - notifying 1 OnShardInitialized callbacks 08:42:04,6 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 08:42:04,7 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:42:04,7 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 08:42:04,7 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:04,7 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 08:42:04,7 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:42:04,7 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:42:04,7 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:42:04,7 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-2-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 08:42:04,9 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: retiring state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=7252915359, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 08:42:04,11 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, nanosAgo=15490, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 08:42:04,12 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 08:42:04,12 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 08:42:04,12 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 373.0 μs 08:42:04,13 AM [cluster-test-shard-dispatcher-16] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-1-txn-0-1 08:42:04,16 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 08:42:04,16 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 08:42:04,16 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 08:42:04,17 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Stopping Shard 08:42:04,17 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal closed 08:42:04,17 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:42:04,17 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:42:04,19 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:04,19 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:04,25 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:04,25 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:04,25 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:04,26 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:04,28 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:42:04,40 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 08:42:04,41 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddShardReplica 08:42:04,41 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 08:42:04,42 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Stopping Shard 08:42:04,42 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddShardReplica: journal closed 08:42:04,43 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:42:04,43 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:42:04,43 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 08:42:04,44 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#889165531] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#889165531] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:04,44 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddShardReplica 08:42:04,44 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#889165531] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#889165531] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:04,44 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 08:42:04,44 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Stopping Shard 08:42:04,45 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddShardReplica: journal closed 08:42:04,45 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:42:04,45 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:42:04,46 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:04,46 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:05,39 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-3267786605008534065] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:42:05,59 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:05,59 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:05,60 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [1260362309976744035] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:42:05,62 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:42:05.064] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:05,68 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 08:42:05,68 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplica 08:42:05,68 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 08:42:05,69 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5ebde9ac 08:42:05,69 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5ebde9ac 08:42:05,69 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: pauseLeader successfully completed - doing transfer 08:42:05,69 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4d6dcf1a 08:42:05,631 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:42:05,631 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:05,631 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:05,632 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 08:42:05,632 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:42:05,632 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 08:42:05,632 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:42:05,632 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 08:42:05,632 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-oper_testAddShardReplica: Peer address for peer member-3-shard-cars-oper_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 08:42:05,632 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-cars-config_testAddShardReplica: Peer address for peer member-3-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:42:06,650 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:42:09,710 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:09,710 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:09,710 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 08:42:11,466 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:42:11,500 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#724221478] to Actor[pekko://cluster-test/deadLetters] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:11,500 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#-919052577] to Actor[pekko://cluster-test/deadLetters] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:11,750 AM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 08:42:11,751 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:11,751 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:11,751 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:42:12,746 AM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:42:14,339 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: leader transfer aborted 08:42:14,339 AM [cluster-test-shard-dispatcher-28] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: Failed to transfer leadership in 9.271 s 08:42:14,339 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In unpauseLeader 08:42:14,343 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Stopping Shard 08:42:14,344 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplica: journal closed 08:42:14,344 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:42:14,345 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:42:14,346 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 08:42:14,346 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplica 08:42:14,346 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 08:42:14,347 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@180ecbad 08:42:14,347 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@36ef66bd 08:42:14,347 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@36ef66bd 08:42:14,347 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: pauseLeader successfully completed - doing transfer 08:42:15,10 AM [cluster-test-shard-dispatcher-28] [WARN] Leader - member-1-shard-cars-oper_testAddShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 08:42:15,10 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:42:15,10 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: leader transfer aborted 08:42:15,10 AM [cluster-test-shard-dispatcher-28] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: Failed to transfer leadership in 663.4 ms 08:42:15,10 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In unpauseLeader 08:42:15,10 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:15,11 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Leader to IsolatedLeader 08:42:15,11 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 08:42:15,11 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Stopping Shard 08:42:15,12 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplica: journal closed 08:42:15,12 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:42:15,12 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:42:15,14 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:15,14 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [08/30/2025 08:42:15.019] [cluster-test-pekko.actor.internal-dispatcher-5] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:15,20 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:42:15,44 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:42:15,55 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-8083995007600588428] 08:42:15,56 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:42:15,60 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:42:15,60 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:42:15,62 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:15,62 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:15,73 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:15,75 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:42:15,75 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:42:15,76 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:42:15,76 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:15,78 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 08:42:15,81 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2046186746] 08:42:15,83 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2046186746] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,84 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:42:15,84 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:42:15,84 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:42:15,84 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,84 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,85 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:15,85 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,85 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,90 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:15,92 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 08:42:15,92 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@65f3c3ce} 08:42:15,92 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:15,92 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,92 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:15,94 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:15,97 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:42:15,99 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,99 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,99 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-374495218] 08:42:15,99 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,100 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-735018028] 08:42:15,100 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:42:15,100 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:42:15,100 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:42:15,100 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:42:15,100 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-932443199 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,100 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,100 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-374495218] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,101 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,101 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:15,101 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,101 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,101 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:42:15,101 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,101 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,101 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1793567933] 08:42:15,101 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#389260870 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,101 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:42:15,101 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:42:15,103 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:42:15,103 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:42:15,106 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-735018028] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,106 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-932443199] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,106 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1793567933] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,106 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#389260870] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,106 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:15,107 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@349f5e0f} 08:42:15,107 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 08:42:15,111 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:15,111 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,112 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,112 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,112 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1095166770] 08:42:15,112 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,112 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:42:15,112 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1095166770] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,112 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,112 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,112 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#764056505] 08:42:15,112 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:42:15,112 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#2012310220 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,113 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:42:15,113 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#764056505] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,113 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#2012310220] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,113 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:42:15,113 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1283299083 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,113 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1283299083] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,113 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:15,113 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:42:15,113 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:42:15,114 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:15,115 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:15,115 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:15,117 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:42:15,117 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,117 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,117 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:42:15,117 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:42:15,117 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:42:15,117 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:42:15,118 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,118 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:42:15,118 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,118 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 08:42:15,118 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:42:15,118 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:42:15,118 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 08:42:15,118 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 08:42:15,118 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:42:15,118 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 08:42:15,118 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 08:42:15,118 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:42:15,118 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:42:15,118 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 08:42:15,118 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,118 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,118 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:42:15,118 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 08:42:15,118 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 08:42:15,122 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,122 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,122 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:42:15,122 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:42:15,122 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 08:42:15,122 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 08:42:15,123 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:42:15,133 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-115364556972766531] 08:42:15,133 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.1.5] ... 08:42:15,136 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:15,136 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:42:15,141 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:15,141 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:15,148 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-1-txn-0-0} 08:42:15,164 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:15,166 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:15,181 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 08:42:15,200 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1099117126] 08:42:15,205 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:42:15,205 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:42:15,205 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:42:15,205 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,206 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,221 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1099117126] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,231 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:15,232 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@6768324a} 08:42:15,232 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 08:42:15,233 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:15,233 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:15,233 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,235 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,235 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:15,235 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#994336864] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,236 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:42:15,236 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#2021330174 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,236 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#2021330174] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,237 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:42:15,239 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:42:15,241 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:42:15,241 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:15,241 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#1618890370] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,241 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:42:15,242 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:42:15,243 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1790222074] 08:42:15,244 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:42:15,244 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1790222074] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,244 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:42:15,244 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:42:15,244 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,244 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,245 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:42:15,245 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#288549085 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,245 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#288549085] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,245 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:42:15,245 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,245 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,246 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:42:15,246 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:42:15,246 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 08:42:15,246 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 08:42:15,246 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:42:15,247 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 08:42:15,247 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:15,247 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,247 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,247 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:42:15,248 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:15,248 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:15,249 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 08:42:15,249 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@1d385f4d} 08:42:15,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:15,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,250 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1731452328] 08:42:15,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,250 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:42:15,250 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#373160961] 08:42:15,251 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:42:15,251 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1731452328] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,251 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:42:15,251 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 08:42:15,251 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:42:15,251 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:42:15,251 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#373160961] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,252 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-835341016] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,252 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:42:15,252 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1414409098 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,252 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:42:15,252 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1414409098] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,253 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:42:15,260 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:15,261 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:15,261 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,261 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,261 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,261 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,261 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,261 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,261 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,261 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,261 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:42:15,261 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 08:42:15,261 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 08:42:15,262 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-835341016 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,262 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:42:15,263 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:42:15,263 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 08:42:15,263 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,263 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,263 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:42:15,263 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,263 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:42:15,263 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 08:42:15,265 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:15,267 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:42:15,268 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:42:15,268 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 08:42:15,268 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:42:15,268 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,268 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,268 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:42:15,270 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,270 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:42:15,270 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 08:42:15,279 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-8040566358331457601] 08:42:15,279 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.1.5] ... 08:42:15,281 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:15,281 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:42:15,285 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:15,285 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:15,291 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:15,293 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:15,294 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 08:42:15,296 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1622046834] 08:42:15,300 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:42:15,300 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:42:15,300 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:42:15,300 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,300 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,301 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1622046834] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,311 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:15,312 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@65613a5e} 08:42:15,312 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 08:42:15,312 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:15,313 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:15,313 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,315 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:15,320 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:42:15,321 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,321 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#218203292] 08:42:15,321 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:42:15,322 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:42:15,322 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:42:15,322 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:42:15,322 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:42:15,322 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,322 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#690212486 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,322 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#47670470] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,322 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#218203292] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,322 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#690212486] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,322 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,323 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:42:15,323 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:42:15,323 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#1472425005] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,323 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-178057494 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,323 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-178057494] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,324 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:42:15,325 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:42:15,326 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:15,328 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:42:15,329 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,329 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,329 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:42:15,330 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:42:15,330 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:42:15,330 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 08:42:15,330 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 08:42:15,331 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 08:42:15,332 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:42:15,334 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:15,334 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:15,334 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,334 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,334 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,334 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,334 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:15,334 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,334 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,334 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:15,334 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:15,336 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@340e72c3} 08:42:15,336 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 08:42:15,336 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:15,336 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,336 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:15,337 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1115505644] 08:42:15,337 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,337 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1115505644] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,337 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:42:15,337 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:42:15,338 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#121965650] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,338 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1546583550 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,338 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1865787122 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,338 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1546583550] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,338 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:42:15,338 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,338 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,338 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#121965650] 08:42:15,338 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1865787122] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:15,339 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:42:15,340 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:42:15,340 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:42:15,341 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:15,341 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:15,342 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:42:15,342 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:42:15,342 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 08:42:15,342 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,343 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,343 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:42:15,343 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:15,343 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 08:42:15,343 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 08:42:15,343 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:42:15,344 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:42:15,344 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 08:42:15,344 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,344 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,344 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:42:15,344 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:15,344 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:42:15,344 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:42:15,344 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 08:42:15,344 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:42:15,344 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 08:42:15,344 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,344 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,344 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:42:15,344 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:15,345 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:42:15,345 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 08:42:15,455 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 08:42:15,455 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:15,455 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,455 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,455 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 08:42:15,456 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 08:42:15,475 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 08:42:15,475 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:15,475 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,475 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,475 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 08:42:15,475 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 08:42:15,485 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 08:42:15,485 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:15,485 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,485 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,485 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 08:42:15,485 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 08:42:15,514 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 08:42:15,514 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:15,514 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:42:15,514 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:15,514 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 08:42:15,514 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 08:42:15,845 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 08:42:15,854 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 08:42:15,905 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 08:42:15,914 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 08:42:16,84 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:42:16,84 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:16,84 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:16,84 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:42:16,84 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:16,84 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:16,84 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,84 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,85 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,85 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,85 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,85 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,85 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,85 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,85 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,85 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,85 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,85 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,91 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:16,91 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:16,91 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,91 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,91 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,91 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,91 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,91 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:16,92 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:16,92 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,92 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:16,168 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:16,168 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:16,168 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,168 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:16,169 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,169 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,169 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,169 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,169 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,169 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,169 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,196 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 08:42:16,197 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 08:42:16,198 AM [cluster-test-shard-dispatcher-35] [INFO] Follower - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:16,199 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:16,199 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:16,199 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:16,208 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:42:16,208 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:42:16,208 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:42:16,208 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:42:16,208 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 08:42:16,208 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 08:42:16,209 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 08:42:16,209 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3541024b 08:42:16,209 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 08:42:16,209 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@26e9c8cf 08:42:16,209 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 08:42:16,209 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 08:42:16,209 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 08:42:16,209 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 08:42:16,210 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3ac98a32 08:42:16,210 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 08:42:16,210 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@31fbbdf4 08:42:16,210 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@507258a4 08:42:16,211 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 08:42:16,211 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4dc09ecf 08:42:16,245 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 08:42:16,246 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:16,247 AM [cluster-test-shard-dispatcher-35] [INFO] Follower - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:16,254 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:42:16,254 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:42:16,254 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 08:42:16,254 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 08:42:16,254 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3b86ea93 08:42:16,254 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 08:42:16,255 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 08:42:16,255 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 08:42:16,255 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:16,256 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@8c21913 08:42:16,256 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 08:42:16,256 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 08:42:16,256 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7693aaae 08:42:16,256 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 08:42:16,257 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 08:42:16,265 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 08:42:16,266 AM [cluster-test-shard-dispatcher-35] [INFO] Follower - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:16,266 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:16,274 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:42:16,274 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:42:16,274 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 08:42:16,274 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 08:42:16,274 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@282e3262 08:42:16,274 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 08:42:16,274 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 08:42:16,274 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 08:42:16,274 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:16,274 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 08:42:16,274 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:16,275 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@85780b9 08:42:16,275 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 08:42:16,275 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 08:42:16,275 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 08:42:16,275 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2046186746] 08:42:16,276 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:16,276 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-374495218] 08:42:16,276 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-374495218] 08:42:16,276 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1099117126] 08:42:16,276 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@33912a18 08:42:16,276 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 08:42:16,277 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:42:16,277 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#994336864] 08:42:16,277 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 08:42:16,277 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#994336864] 08:42:16,277 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1622046834] 08:42:16,277 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:42:16,277 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#47670470] 08:42:16,277 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#47670470] 08:42:16,287 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for all shards 08:42:16,287 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testChangeMemberVotingStatesForAllShards 08:42:16,288 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testChangeMemberVotingStatesForAllShards 08:42:16,288 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 08:42:16,288 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 08:42:16,290 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 08:42:16,290 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 08:42:16,291 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:42:16,291 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:42:16,291 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:42:16,291 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:42:16,292 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,292 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,292 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,295 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,296 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:42:16,296 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:42:16,297 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:42:16,298 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,298 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 08:42:16,298 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:42:16,298 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 08:42:16,298 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,298 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 08:42:16,298 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:42:16,298 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 08:42:16,299 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,299 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 08:42:16,299 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:42:16,299 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 08:42:16,299 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:42:16,301 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,301 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 08:42:16,301 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:42:16,301 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 08:42:16,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2046186746] 08:42:16,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:16,303 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-374495218] 08:42:16,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-374495218] 08:42:16,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2046186746] 08:42:16,304 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:42:16,304 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1793567933] 08:42:16,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#1793567933] 08:42:16,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-735018028] 08:42:16,304 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:42:16,304 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1095166770] 08:42:16,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1095166770] 08:42:16,305 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-735018028] 08:42:16,305 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:42:16,305 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#764056505] 08:42:16,305 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#764056505] 08:42:16,305 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:42:16,305 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1099117126] 08:42:16,305 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:42:16,305 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#994336864] 08:42:16,305 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#994336864] 08:42:16,306 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 08:42:16,307 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 08:42:16,308 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 08:42:16,308 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1099117126] 08:42:16,308 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:42:16,308 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#1618890370] 08:42:16,308 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 08:42:16,308 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 08:42:16,308 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#1618890370] 08:42:16,308 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 08:42:16,309 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 08:42:16,309 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1790222074] 08:42:16,309 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:42:16,309 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1731452328] 08:42:16,309 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1731452328] 08:42:16,309 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1790222074] 08:42:16,310 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 08:42:16,310 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#373160961] 08:42:16,310 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 08:42:16,310 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#373160961] 08:42:16,311 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1622046834] 08:42:16,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:42:16,311 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#47670470] 08:42:16,311 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#47670470] 08:42:16,311 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1622046834] 08:42:16,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:42:16,311 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#1472425005] 08:42:16,311 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#1472425005] 08:42:16,311 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#218203292] 08:42:16,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:42:16,311 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1115505644] 08:42:16,312 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-1115505644] 08:42:16,312 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#218203292] 08:42:16,312 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:42:16,312 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#121965650] 08:42:16,312 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#121965650] 08:42:16,312 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:16,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,313 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,313 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 08:42:16,313 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:42:16,313 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:42:16,313 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 08:42:16,313 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 08:42:16,314 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:42:16,314 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:42:16,314 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:16,314 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,314 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,314 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:42:16,314 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:42:16,315 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:42:16,315 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 08:42:16,315 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 08:42:16,315 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:42:16,315 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:42:16,316 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:16,316 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:16,320 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:16,320 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:16,320 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:16,320 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:16,322 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:42:16,336 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:16,337 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:16,337 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:16,337 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:42:16,338 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:42:16,338 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:42:16,339 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 08:42:16,339 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 08:42:16,340 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:42:16,340 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:42:16,340 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:16,341 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:16,341 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:16,341 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:42:16,341 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:42:16,341 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:42:16,341 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 08:42:16,341 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 08:42:16,342 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:42:16,342 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:42:16,343 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:16,343 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:17,103 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:42:17,332 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-8040566358331457601] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:42:17,362 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-115364556972766531] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:42:17,363 AM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:17,363 AM [cluster-test-pekko.actor.default-dispatcher-33] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:17,365 AM [cluster-test-pekko.actor.default-dispatcher-33] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:42:17.367] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:17,382 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:17,382 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:42:17,382 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:42:17,382 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:42:17,382 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2918d82f 08:42:17,383 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@729ef8cd 08:42:17,383 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@729ef8cd 08:42:17,383 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 08:42:17,383 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6373ec0d 08:42:17,383 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4666dcd4 08:42:17,383 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4666dcd4 08:42:17,383 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 08:42:17,802 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 08:42:17,802 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 420.0 ms 08:42:17,802 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 08:42:17,803 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:42:17,804 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 08:42:17,852 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 08:42:17,852 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 469.2 ms 08:42:17,852 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 08:42:17,853 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:42:17,853 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 08:42:17,854 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:42:17,854 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:42:17,854 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:42:17,855 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:42:17,855 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:42:17,855 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:42:17,855 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@248ba98b 08:42:17,855 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3644a7c 08:42:17,855 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3644a7c 08:42:17,855 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 08:42:17,855 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1edae1da 08:42:17,855 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@74de8337 08:42:17,856 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@74de8337 08:42:17,856 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 08:42:18,192 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 08:42:18,192 AM [cluster-test-shard-dispatcher-24] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 337.0 ms 08:42:18,192 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 08:42:18,193 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:42:18,194 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 08:42:18,441 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 08:42:18,441 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 586.3 ms 08:42:18,441 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 08:42:18,442 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:42:18,443 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 08:42:18,443 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:42:18,443 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:42:18,445 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:18,445 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:18,448 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:42:18.450] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:18,486 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:42:18,503 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [7198328108574987062] 08:42:18,503 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:42:18,504 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:42:18,504 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:42:18,505 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:18,505 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:18,526 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,527 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:42:18,528 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:42:18,528 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,530 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:42:18,530 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testGetPrefixShardRole 08:42:18,531 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-491617360] 08:42:18,532 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:42:18,532 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:42:18,532 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:42:18,532 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-default-config_testGetPrefixShardRole 08:42:18,533 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:18,533 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-491617360] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,533 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 08:42:18,539 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 08:42:18,541 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@4bf49716} 08:42:18,541 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} started 08:42:18,541 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:18,541 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config_testGetPrefixShardRole 08:42:18,541 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,542 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 08:42:18,542 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 08:42:18,542 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-default-config_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole on shard member-1-shard-default-config_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-843878673] 08:42:18,542 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-843878673] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,543 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Shard created, persistent : true 08:42:18,543 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,543 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-456250199 created and ready for shard:member-1-shard-default-config_testGetPrefixShardRole 08:42:18,544 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-456250199] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,544 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-default-config_testGetPrefixShardRole: Starting recovery with journal batch size 1 08:42:18,546 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testGetPrefixShardRole 08:42:18,546 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-default-config_testGetPrefixShardRole: journal open: applyTo=0 08:42:18,547 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#1348766430] 08:42:18,547 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:42:18,548 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:42:18,548 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:42:18,548 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-default-oper_testGetPrefixShardRole 08:42:18,549 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:18,549 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 08:42:18,551 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#1348766430] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,552 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-default-config_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,556 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-default-config_testGetPrefixShardRole: Local TermInfo store seeded with TermInfo{term=0} 08:42:18,556 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [default] 08:42:18,556 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-default-config_testGetPrefixShardRole is initialized 08:42:18,556 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,557 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 08:42:18,557 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from null to Follower 08:42:18,558 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@5ab5cc52} 08:42:18,558 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} started 08:42:18,558 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:18,558 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-default-oper_testGetPrefixShardRole 08:42:18,559 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Starting new election term 1 08:42:18,559 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-default-config_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:18,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,559 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 08:42:18,559 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 08:42:18,559 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-default-config_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:42:18,559 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-default-oper_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole on shard member-1-shard-default-oper_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-1997611927] 08:42:18,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 08:42:18,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: became leader with frontend state for [] 08:42:18,560 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Shard created, persistent : true 08:42:18,560 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-1997611927] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,561 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#2076915037 created and ready for shard:member-1-shard-default-oper_testGetPrefixShardRole 08:42:18,561 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#2076915037] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,561 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-default-oper_testGetPrefixShardRole: Starting recovery with journal batch size 1 08:42:18,562 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-default-oper_testGetPrefixShardRole: journal open: applyTo=0 08:42:18,559 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Follower to Candidate 08:42:18,564 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Candidate to Leader 08:42:18,564 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole 08:42:18,564 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@747ad8f7 08:42:18,564 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-default-config_testGetPrefixShardRole from Candidate to Leader 08:42:18,564 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testGetPrefixShardRole is ready 08:42:18,565 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-default-oper_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,567 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-default-oper_testGetPrefixShardRole: Local TermInfo store seeded with TermInfo{term=0} 08:42:18,568 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,568 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [default] 08:42:18,575 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,575 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-default-oper_testGetPrefixShardRole is initialized 08:42:18,576 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from null to Follower 08:42:18,578 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Starting new election term 1 08:42:18,578 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-default-oper_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:18,578 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,578 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,578 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Follower to Candidate 08:42:18,578 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-default-oper_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:42:18,578 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 08:42:18,578 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: became leader with frontend state for [] 08:42:18,579 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Candidate to Leader 08:42:18,579 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole 08:42:18,579 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1bfa4a63 08:42:18,580 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-default-oper_testGetPrefixShardRole from Candidate to Leader 08:42:18,580 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testGetPrefixShardRole is ready 08:42:18,609 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-491617360] 08:42:18,609 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : default 08:42:18,609 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-843878673] 08:42:18,610 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 08:42:18,610 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-default-config_testGetPrefixShardRole 08:42:18,610 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 08:42:18,610 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@40e6a23 08:42:18,610 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-default-config_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@40e6a23 08:42:18,611 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Stopping Shard 08:42:18,611 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-default-config_testGetPrefixShardRole: journal closed 08:42:18,611 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:42:18,612 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:42:18,612 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 08:42:18,612 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-default-oper_testGetPrefixShardRole 08:42:18,612 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 08:42:18,612 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@6bad1193 08:42:18,612 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-default-oper_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@6bad1193 08:42:18,612 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-491617360] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-491617360] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,612 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f0d54803ba0] to Actor[pekko://cluster-test/user/$a#1724322366] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#1724322366] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,612 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-491617360] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-491617360] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,612 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Stopping Shard 08:42:18,613 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-default-oper_testGetPrefixShardRole: journal closed 08:42:18,613 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:42:18,613 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:42:18,614 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#1348766430] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#1348766430] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,614 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:18,615 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:18,724 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:42:18,739 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [8834364946258125486] 08:42:18,739 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:42:18,740 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:42:18,740 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:42:18,741 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:18,741 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:18,752 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,754 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,756 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,757 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:42:18,757 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:42:18,757 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:42:18,758 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1392042885] 08:42:18,758 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1392042885] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,758 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:42:18,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:42:18,759 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:42:18,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,759 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,760 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:18,760 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,760 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,763 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:42:18,763 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@4c0811ad} 08:42:18,763 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 08:42:18,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:18,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,764 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,764 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#64401157] 08:42:18,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,765 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,765 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,765 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-13118506] 08:42:18,765 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:42:18,765 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,765 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:42:18,767 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:42:18,767 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:42:18,767 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#941021305 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,767 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#64401157] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,767 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-13118506] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,767 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#941021305] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,767 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1518952297 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,767 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:42:18,767 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1518952297] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,772 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:42:18,773 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,774 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,774 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,775 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,775 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,775 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:42:18,775 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,775 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,775 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:42:18,775 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:42:18,775 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:42:18,775 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:42:18,775 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:42:18,775 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,775 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,775 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:42:18,775 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:42:18,776 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-626886033] 08:42:18,777 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-626886033] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,778 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:42:18,778 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:42:18,778 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:42:18,778 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,778 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,778 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:18,778 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,778 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,779 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:42:18,780 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@787214d0} 08:42:18,780 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 08:42:18,781 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:18,781 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,781 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:42:18,782 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:42:18,783 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:42:18,783 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,784 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,784 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1064699449] 08:42:18,784 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,784 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,784 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,784 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#103143145] 08:42:18,784 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:42:18,785 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1078886062 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,785 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1500311721 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,785 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:42:18,786 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,786 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,786 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:42:18,786 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,786 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:42:18,786 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:42:18,786 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,786 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:42:18,788 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1064699449] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,788 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#103143145] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,788 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1078886062] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,788 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1500311721] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,788 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:42:18,789 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,790 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:42:18,790 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:42:18,790 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,790 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,790 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:42:18,790 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,790 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:42:18,792 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:42:18,803 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-2376595110264675341] 08:42:18,804 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.1.5] ... 08:42:18,806 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:18,806 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:42:18,806 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:18,806 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:18,820 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,822 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,824 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,837 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1974412893] 08:42:18,837 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1974412893] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,838 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:42:18,838 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:42:18,838 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:42:18,838 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,839 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,842 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:42:18,842 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@4c3e5e84} 08:42:18,842 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 08:42:18,843 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:18,843 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,843 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,845 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,847 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,848 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:42:18,848 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,849 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:42:18,850 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:42:18,851 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:42:18,851 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:42:18,853 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:42:18,853 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1487330605] 08:42:18,854 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#2013230430] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,854 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1976523365] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,854 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1487330605] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,855 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:42:18,855 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:42:18,855 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:42:18,855 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,855 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,856 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#433465364 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,856 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1257258976] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,856 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#433465364] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,860 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1257258976 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,860 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,861 AM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,861 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,861 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,861 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:42:18,861 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:42:18,861 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:42:18,861 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:42:18,861 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:42:18,861 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,862 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:42:18,862 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,862 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,862 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:42:18,862 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,862 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:42:18,864 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:42:18,865 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 08:42:18,865 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@3483d3f9} 08:42:18,866 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:18,866 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,866 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,867 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:42:18,867 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:42:18,868 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:42:18,869 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#390511434] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,869 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:42:18,869 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-705851128] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,869 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-705851128 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,870 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:42:18,871 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:42:18,871 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1851912082] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,872 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#23012142] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,872 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#23012142 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,872 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:42:18,876 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,878 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:42:18,878 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:42:18,878 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,878 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,878 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:42:18,879 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,880 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,880 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:42:18,881 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:18,881 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,881 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:42:18,881 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,881 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:18,881 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,881 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,881 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,881 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,881 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,881 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:42:18,881 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:42:18,881 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,882 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,882 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,882 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,882 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,882 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,882 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:42:18,882 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,883 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,883 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:42:18,884 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:42:18,894 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-4742086602678301129] 08:42:18,895 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.1.5] ... 08:42:18,897 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:18,897 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:42:18,900 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:18,900 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:18,910 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,911 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,913 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,916 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1158571643] 08:42:18,916 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:42:18,916 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:42:18,916 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:42:18,916 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,917 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,920 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1158571643] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,924 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:42:18,925 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@657bf872} 08:42:18,925 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 08:42:18,925 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,927 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:18,929 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,929 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-235329695] 08:42:18,930 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-235329695] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,930 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:42:18,930 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:42:18,930 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:42:18,930 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,931 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,931 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:18,931 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,932 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-806692503] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,932 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:42:18,932 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#953763631 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,932 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#953763631] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,933 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:42:18,934 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:42:18,935 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,937 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:42:18,937 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:42:18,938 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:42:18,938 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,938 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:42:18,938 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:42:18,938 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1185529203] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,938 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1520939402 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,938 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1520939402] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,938 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:42:18,938 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:42:18,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@be45322} 08:42:18,939 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 08:42:18,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:18,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,940 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:42:18,941 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:42:18,941 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:42:18,941 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:42:18,941 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1242747696] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,942 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1027983623 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,942 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1027983623] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,942 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,943 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,943 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,943 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:42:18,943 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,944 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,944 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,944 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:42:18,944 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:42:18,944 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:42:18,944 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,944 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:42:18,945 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-635078905] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,945 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:42:18,945 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1878724727 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,945 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1878724727] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:18,946 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:42:18,947 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:42:18,947 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:42:18,948 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:42:18,948 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,948 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:18,948 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:18,948 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:18,948 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:42:18,949 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:42:18,949 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:42:18,949 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:42:18,949 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,949 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:42:18,961 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:42:18,964 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:42:18,965 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:18,965 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,965 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,965 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,965 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,965 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,965 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,965 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:18,965 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,965 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,965 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,965 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,965 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:18,965 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,757 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:42:19,757 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:42:19,758 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:19,758 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,758 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:19,759 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,759 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,759 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,759 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,759 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,759 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,759 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,759 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:19,766 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:19,766 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:19,766 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:19,766 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,766 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,840 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:19,840 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,840 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,840 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,840 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,840 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,841 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,842 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1392042885] 08:42:19,842 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:19,842 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#64401157] 08:42:19,842 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#64401157] 08:42:19,842 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1392042885] 08:42:19,842 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:19,843 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#64401157] 08:42:19,843 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#64401157] 08:42:19,843 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 08:42:19,843 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,843 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,843 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:42:19,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:42:19,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:42:19,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:42:19,843 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:42:19,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:42:19,844 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:19,844 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:42:19,844 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,844 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,845 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,845 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,848 AM [cluster-test-shard-dispatcher-30] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 08:42:19,848 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:42:19,848 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:19,848 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:19,849 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 08:42:19,849 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 08:42:19,849 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 08:42:19,849 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 08:42:19,849 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 08:42:19,849 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:42:19,849 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:19,849 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:42:19,849 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:19,849 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:19,849 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:19,849 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 08:42:19,849 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:42:19,849 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 08:42:19,849 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:42:19,849 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:19,849 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 08:42:19,849 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 08:42:19,849 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 08:42:19,849 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 08:42:19,850 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:19,850 AM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:19,850 AM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:19,850 AM [cluster-test-shard-dispatcher-24] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:19,850 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:19,851 AM [cluster-test-shard-dispatcher-35] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:19,851 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:19,851 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:19,856 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:42:19,856 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:42:19,856 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:42:19,857 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 08:42:19,857 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@14112c62 08:42:19,857 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,857 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 08:42:19,857 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:42:19,857 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 08:42:19,857 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 08:42:19,857 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 08:42:19,858 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@529a77ab 08:42:19,858 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:42:19,858 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:42:19,859 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@713e73c1 08:42:19,859 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:42:19,859 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:42:19,860 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:42:19,860 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:42:19,860 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:42:19,860 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:42:19,860 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 08:42:19,860 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:42:19,860 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:42:19,860 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 08:42:19,861 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:42:19,861 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 08:42:19,861 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:42:19,861 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,862 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:42:19,862 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 08:42:19,862 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 08:42:19,862 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:42:19,862 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@422174a7 08:42:19,862 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 08:42:19,861 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 08:42:19,861 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,862 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 08:42:19,861 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@10c32e97 08:42:19,862 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:42:19,862 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 08:42:19,862 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 08:42:19,862 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@43bc3026 08:42:19,862 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,862 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 08:42:19,862 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 08:42:19,862 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:42:19,862 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 08:42:19,862 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 08:42:19,862 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:42:19,862 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:42:19,862 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1392042885] 08:42:19,863 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:19,863 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 08:42:19,863 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#64401157] 08:42:19,863 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#64401157] 08:42:19,864 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1392042885] 08:42:19,864 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:42:19,864 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-13118506] 08:42:19,864 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-13118506] 08:42:19,864 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:42:19,865 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1d894232 08:42:19,865 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7ff99396 08:42:19,865 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 08:42:19,865 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:42:19,865 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:42:19,865 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:42:19,865 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:42:19,865 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c165a07 08:42:19,865 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:42:19,865 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@52d1bff5 08:42:19,865 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@eee99f3 08:42:19,865 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 08:42:19,865 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 08:42:19,866 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 08:42:19,866 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:42:19,866 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:42:19,866 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:42:19,866 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:42:19,866 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ea4e40a 08:42:19,866 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 08:42:19,866 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:42:19,866 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-626886033] 08:42:19,866 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:42:19,866 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 08:42:19,866 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:42:19,866 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 08:42:19,867 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1064699449] 08:42:19,867 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1064699449] 08:42:19,867 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-626886033] 08:42:19,867 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:42:19,869 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#103143145] 08:42:19,869 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#103143145] 08:42:19,869 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1974412893] 08:42:19,869 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:42:19,869 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#2013230430] 08:42:19,869 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#2013230430] 08:42:19,872 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 08:42:19,872 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 08:42:19,872 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:42:19,872 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:42:19,872 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 08:42:19,873 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:42:19,873 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:42:19,873 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:42:19,873 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:42:19,873 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 08:42:19,873 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 08:42:19,873 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 08:42:19,873 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 08:42:19,873 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 08:42:19,873 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 08:42:19,873 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 08:42:19,873 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 08:42:19,919 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:42:19,920 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#2013230430] 08:42:19,920 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1974412893] 08:42:19,920 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:42:19,920 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1976523365] 08:42:19,921 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1976523365] 08:42:19,921 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1487330605] 08:42:19,921 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:42:19,921 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#390511434] 08:42:19,921 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#390511434] 08:42:19,921 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1487330605] 08:42:19,921 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 08:42:19,922 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1851912082] 08:42:19,922 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1851912082] 08:42:19,922 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1158571643] 08:42:19,922 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:42:19,922 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-806692503] 08:42:19,922 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-806692503] 08:42:19,923 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1158571643] 08:42:19,923 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:42:19,923 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1185529203] 08:42:19,923 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1185529203] 08:42:19,923 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-235329695] 08:42:19,923 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:42:19,923 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1242747696] 08:42:19,923 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1242747696] 08:42:19,924 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-235329695] 08:42:19,924 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:42:19,924 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-635078905] 08:42:19,924 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-635078905] 08:42:19,924 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1392042885] 08:42:19,925 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:19,925 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#64401157] 08:42:19,925 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#64401157] 08:42:19,925 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-626886033] 08:42:19,926 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:42:19,926 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1064699449] 08:42:19,926 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1064699449] 08:42:19,926 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:42:19,927 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,927 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,927 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 08:42:19,927 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:42:19,927 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:42:19,928 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:42:19,928 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:42:19,928 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:42:19,928 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:42:19,929 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:42:19,929 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,929 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,930 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:42:19,930 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:42:19,930 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:42:19,930 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:42:19,930 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:42:19,931 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:42:19,931 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:42:19,932 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:19,932 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:19,936 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:19,936 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:19,936 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:19,936 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:19,938 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:42:19,950 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:42:19,951 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,951 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,951 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:42:19,951 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:42:19,951 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:42:19,951 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:42:19,952 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:42:19,952 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:42:19,952 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:42:19,953 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:42:19,953 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,954 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:19,954 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:42:19,954 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:42:19,954 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:42:19,955 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:42:19,955 AM [cluster-test-shard-dispatcher-35] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:42:19,955 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:42:19,955 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:42:19,956 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:19,957 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:20,777 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:42:20,946 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-4742086602678301129] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:42:20,976 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:20,977 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:20,977 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-2376595110264675341] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:42:20,978 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:42:20.979] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:20,986 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:42:20,986 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:20,987 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:42:20,987 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:42:20,987 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@b00e3f5 08:42:20,987 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@b00e3f5 08:42:20,987 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 08:42:20,987 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ce68293 08:42:20,987 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2a6d016b 08:42:20,987 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@48d91fd7 08:42:20,987 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2a6d016b 08:42:20,987 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 08:42:21,467 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 08:42:21,467 AM [cluster-test-shard-dispatcher-30] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 08:42:21,467 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 480.0 ms 08:42:21,467 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 08:42:21,467 AM [cluster-test-shard-dispatcher-30] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 480.2 ms 08:42:21,467 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 08:42:21,468 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:42:21,468 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:42:21,469 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:42:21,469 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:42:21,469 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:42:21,469 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:42:21,470 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:42:21,471 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:21,471 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:42:21,471 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:42:21,471 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@ddb88a8 08:42:21,471 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1e1f199f 08:42:21,471 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1e1f199f 08:42:21,472 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 08:42:21,472 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5ffa8cb9 08:42:21,472 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5ffa8cb9 08:42:21,472 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 08:42:21,472 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@56c4b647 08:42:21,776 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 08:42:21,776 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 305.3 ms 08:42:21,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 08:42:21,777 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:42:21,778 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:42:21,786 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 08:42:21,786 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 315.1 ms 08:42:21,787 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 08:42:21,787 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:42:21,788 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:42:21,788 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:42:21,788 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:42:21,789 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:21,789 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:21,792 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:42:21.793] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:21,880 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:42:21,890 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [7313569537293427197] 08:42:21,893 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:42:21,893 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:42:21,893 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:42:21,894 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:21,894 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:21,906 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:21,907 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:21,909 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,910 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:42:21,910 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:42:21,910 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:42:21,910 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1918982429] 08:42:21,911 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:42:21,911 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:42:21,911 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:42:21,911 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,911 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,911 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:21,911 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,911 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,913 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1918982429] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:21,920 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:21,921 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@2494d92} 08:42:21,921 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 08:42:21,921 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:21,921 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,921 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:21,922 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,922 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,922 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#671103216] 08:42:21,922 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,922 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#671103216] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:21,922 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,922 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,922 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1458490266] 08:42:21,923 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:42:21,923 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:21,923 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1458490266] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:21,923 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:42:21,923 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#234722549 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,924 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#234722549] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:21,924 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1251244995 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,924 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:42:21,924 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:42:21,924 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,925 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:42:21,927 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:21,928 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:42:21,928 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-747237423] 08:42:21,928 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:42:21,928 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:42:21,929 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:42:21,929 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:42:21,929 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:42:21,929 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,929 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,929 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:21,930 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:42:21,930 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:42:21,930 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:21,930 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:21,930 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:42:21,931 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,931 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:42:21,935 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1251244995] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:21,935 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-747237423] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:21,936 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:21,936 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,936 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:21,936 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:21,936 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:42:21,936 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,936 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:42:21,941 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:21,942 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 08:42:21,945 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@7148d05e} 08:42:21,945 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:21,945 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,946 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,946 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,946 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#599930094] 08:42:21,946 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,946 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,946 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,946 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-819268283] 08:42:21,946 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:42:21,946 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#599930094] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:21,946 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:42:21,946 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-819268283] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:21,947 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1714822316 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,947 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1822382860 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,947 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:42:21,947 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:42:21,948 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:42:21,948 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:42:21,950 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:21,950 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1714822316] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:21,950 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:21,950 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:21,950 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1822382860] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:21,951 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:21,951 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:42:21,951 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:42:21,951 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:42:21,951 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,951 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:42:21,951 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:42:21,951 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:21,951 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:42:21,951 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:21,951 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:42:21,951 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:21,951 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:42:21,955 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:42:21,969 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [5818118620461798962] 08:42:21,969 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.1.5] ... 08:42:21,972 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:21,972 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:42:21,973 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:21,973 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:22,8 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:22,10 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:22,37 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,56 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-378060761] 08:42:22,57 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-378060761] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,57 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:42:22,57 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:42:22,57 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:42:22,58 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,58 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,62 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:22,69 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 08:42:22,69 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:22,71 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:22,73 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,74 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@27ca46a3} 08:42:22,74 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:22,74 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,75 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1659153827] 08:42:22,75 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:42:22,75 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:42:22,75 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:42:22,75 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,75 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,76 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1659153827] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,76 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1877112877] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,76 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:42:22,76 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1919487880 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,76 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1919487880] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,77 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,77 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:42:22,78 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#489046593] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,78 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1883764674 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,78 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1883764674] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,79 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:42:22,80 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:22,81 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:42:22,81 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@290dbc2a} 08:42:22,81 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 08:42:22,81 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:22,81 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,82 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,82 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:42:22,82 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:42:22,83 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:42:22,83 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:42:22,84 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1146247237] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,84 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1586606789 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,84 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1395962040] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,84 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1586606789] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,84 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-867361104 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,84 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-867361104] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,85 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:42:22,87 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:42:22,88 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:42:22,88 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:42:22,89 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:22,90 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:42:22,90 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:42:22,90 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:22,90 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:22,90 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:42:22,90 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,90 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:42:22,90 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:22,91 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:22,91 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:42:22,91 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:22,91 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:42:22,91 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:42:22,91 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,92 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:42:22,92 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:22,92 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:42:22,92 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:42:22,92 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:22,92 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:22,92 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:22,92 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:42:22,92 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:42:22,92 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:22,92 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:22,93 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:42:22,93 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,93 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:42:22,93 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:42:22,93 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,93 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:42:22,95 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:42:22,102 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:42:22,107 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-1564281863723761969] 08:42:22,107 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.1.5] ... 08:42:22,110 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:42:22,110 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:22,110 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:42:22,110 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:22,110 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:22,111 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:22,111 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,111 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,111 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,111 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,111 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,111 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,111 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:22,111 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,111 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,111 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,111 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,111 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,111 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,122 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:22,123 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:22,125 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,128 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1811552161] 08:42:22,129 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:42:22,129 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:42:22,129 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:42:22,129 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,129 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,130 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1811552161] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,139 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:22,143 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@2c278acc} 08:42:22,143 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 08:42:22,144 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:42:22,144 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:22,145 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:22,145 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,145 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,145 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:22,146 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:42:22,146 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:42:22,147 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-727926440 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,147 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1887998078] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,148 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:42:22,148 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:42:22,148 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:42:22,149 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,149 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-7304381 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,149 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:42:22,150 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#409598430] 08:42:22,150 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:42:22,150 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:42:22,150 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:42:22,150 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:42:22,150 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,150 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,153 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:22,154 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:42:22,154 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:42:22,154 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:22,154 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:22,154 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:42:22,154 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,154 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:42:22,156 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-727926440] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,156 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-688615999] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,156 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-7304381] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,156 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#409598430] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,157 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:22,157 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,157 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,157 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,157 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,157 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,157 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:22,158 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:22,158 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:22,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:42:22,158 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:42:22,158 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:42:22,158 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,158 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:42:22,159 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:22,159 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,159 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,160 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:22,161 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 08:42:22,161 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:22,161 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@4c938631} 08:42:22,161 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:22,161 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,162 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,162 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,162 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1608621637] 08:42:22,162 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,162 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:42:22,162 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1608621637] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,162 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1371304691 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,162 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:42:22,162 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1371304691] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,163 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,163 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,163 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1118890747] 08:42:22,163 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1118890747] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,163 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:42:22,164 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1649753663 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,164 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1649753663] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:22,164 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:42:22,165 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:42:22,163 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:42:22,166 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:22,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:22,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:22,166 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:42:22,166 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:42:22,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:42:22,167 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,167 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:22,167 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:42:22,167 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:42:22,167 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:42:22,167 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:22,167 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:22,167 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:42:22,167 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,168 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,168 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:42:22,280 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 08:42:22,280 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:42:22,280 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:22,280 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:22,280 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 08:42:22,281 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 08:42:22,289 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 08:42:22,289 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:42:22,290 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:22,290 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:22,290 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 08:42:22,290 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 08:42:22,320 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 08:42:22,320 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:42:22,320 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:22,320 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:22,320 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 08:42:22,320 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 08:42:22,330 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 08:42:22,330 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:42:22,331 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:42:22,331 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:22,331 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 08:42:22,331 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 08:42:22,621 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 08:42:22,621 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 08:42:22,650 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 08:42:22,730 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 08:42:22,909 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:22,909 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,909 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,909 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,909 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,909 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:22,909 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,909 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,909 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,909 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,909 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,909 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:42:22,909 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,909 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,909 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:42:22,909 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,910 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:22,910 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,910 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,910 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,910 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,910 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,910 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,911 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:22,911 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,911 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,911 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,911 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,911 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,911 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,916 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:22,916 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,916 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,916 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,916 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,916 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:22,916 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,916 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,916 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,916 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,916 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,916 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,916 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:22,917 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,917 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,917 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,917 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,917 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,917 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:22,917 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,917 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,917 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,917 AM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,917 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:22,951 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 08:42:22,953 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:22,953 AM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:22,960 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:42:22,961 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 08:42:22,961 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:42:22,961 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 08:42:22,961 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3166f21b 08:42:22,962 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:42:22,962 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:42:22,962 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@56219931 08:42:22,962 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@485260b8 08:42:22,962 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 08:42:22,962 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 08:42:22,964 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:42:22,964 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:42:22,969 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 08:42:22,969 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:42:22,969 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 08:42:22,969 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 08:42:23,1 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:23,1 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,1 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,1 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,2 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,20 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 08:42:23,21 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:23,21 AM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:23,30 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:42:23,30 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 08:42:23,30 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:42:23,30 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 08:42:23,30 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 08:42:23,30 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@127de684 08:42:23,30 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 08:42:23,32 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:42:23,32 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7fd11670 08:42:23,32 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:42:23,32 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:42:23,32 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5cd9f241 08:42:23,32 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:42:23,35 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 08:42:23,35 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:42:23,35 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 08:42:23,35 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 08:42:23,40 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 08:42:23,41 AM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:23,41 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:23,47 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:42:23,48 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 08:42:23,48 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:42:23,48 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 08:42:23,48 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 08:42:23,48 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7f8f2007 08:42:23,48 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 08:42:23,49 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:42:23,49 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@43e98da5 08:42:23,49 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 08:42:23,49 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:42:23,50 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 08:42:23,50 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:42:23,50 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c91a803 08:42:23,50 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 08:42:23,50 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:42:23,51 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:23,51 AM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:42:23,53 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 08:42:23,53 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:42:23,53 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 08:42:23,54 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 08:42:23,54 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 08:42:23,57 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:42:23,57 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 08:42:23,57 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:42:23,57 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 08:42:23,57 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 08:42:23,57 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@d0f3985 08:42:23,57 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 08:42:23,58 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@78272c56 08:42:23,58 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 08:42:23,58 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:42:23,59 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:42:23,60 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 08:42:23,60 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:42:23,60 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 08:42:23,61 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 08:42:23,61 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 08:42:23,61 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:42:23,61 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 08:42:23,61 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:42:23,61 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:23,61 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 08:42:23,61 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7cd07a6c 08:42:23,61 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1918982429] 08:42:23,61 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 08:42:23,61 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:23,61 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#671103216] 08:42:23,61 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#671103216] 08:42:23,61 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 08:42:23,61 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,62 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,62 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:42:23,62 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:42:23,62 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:42:23,62 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:42:23,62 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:42:23,62 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:42:23,62 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:23,62 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:42:23,64 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,64 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,64 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,64 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,64 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 08:42:23,65 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:42:23,65 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 08:42:23,65 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,65 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 08:42:23,65 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:42:23,65 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 08:42:23,65 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,65 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 08:42:23,65 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:42:23,65 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 08:42:23,66 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,67 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,67 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 08:42:23,67 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:42:23,67 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 08:42:23,67 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1918982429] 08:42:23,67 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:23,67 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#671103216] 08:42:23,68 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#671103216] 08:42:23,68 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1918982429] 08:42:23,71 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:42:23,72 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1458490266] 08:42:23,72 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1458490266] 08:42:23,72 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-747237423] 08:42:23,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:42:23,72 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#599930094] 08:42:23,72 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#599930094] 08:42:23,73 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-747237423] 08:42:23,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:42:23,73 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-819268283] 08:42:23,73 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-819268283] 08:42:23,73 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-378060761] 08:42:23,73 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:42:23,73 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1877112877] 08:42:23,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-1877112877] 08:42:23,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-378060761] 08:42:23,74 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:42:23,74 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#489046593] 08:42:23,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#489046593] 08:42:23,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1659153827] 08:42:23,74 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:42:23,74 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1146247237] 08:42:23,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1146247237] 08:42:23,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1659153827] 08:42:23,74 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 08:42:23,75 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1395962040] 08:42:23,75 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1395962040] 08:42:23,75 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1811552161] 08:42:23,75 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:42:23,75 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1887998078] 08:42:23,75 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1887998078] 08:42:23,75 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1811552161] 08:42:23,75 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:42:23,75 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-688615999] 08:42:23,75 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-688615999] 08:42:23,75 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#409598430] 08:42:23,75 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:42:23,75 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1608621637] 08:42:23,75 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1608621637] 08:42:23,76 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#409598430] 08:42:23,76 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:42:23,76 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1118890747] 08:42:23,76 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1118890747] 08:42:23,76 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1918982429] 08:42:23,77 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:23,77 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#671103216] 08:42:23,77 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#671103216] 08:42:23,77 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:23,78 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,78 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,78 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 08:42:23,78 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:42:23,78 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:42:23,79 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:42:23,79 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:42:23,79 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:42:23,79 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:42:23,79 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:23,80 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,80 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,80 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:42:23,80 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:42:23,80 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:42:23,80 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:42:23,80 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:42:23,81 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:42:23,81 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:42:23,81 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:23,82 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:23,87 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:23,88 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:23,88 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:23,89 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:23,91 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:42:23,96 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:23,96 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,96 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,96 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:42:23,97 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:42:23,97 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:42:23,97 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:42:23,97 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:42:23,99 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:42:23,100 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:42:23,102 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:23,103 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,103 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:23,103 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:42:23,103 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:42:23,103 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:42:23,103 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:42:23,104 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:42:23,104 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:42:23,104 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:42:23,105 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:23,105 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:23,677 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 08:42:23,677 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 08:42:23,678 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 08:42:23,678 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 08:42:23,928 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:42:24,97 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-1564281863723761969] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:42:24,118 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [5818118620461798962] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:42:24,127 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:24,127 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:24,129 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:42:24.131] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:24,136 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:24,137 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:24,137 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:42:24,137 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:42:24,138 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7bb0bb2e 08:42:24,138 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7bb0bb2e 08:42:24,138 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 08:42:24,138 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e3a35ad 08:42:24,138 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7e3a35ad 08:42:24,138 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 08:42:24,138 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3438fd23 08:42:24,138 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@701af025 08:42:24,567 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 08:42:24,567 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 429.8 ms 08:42:24,567 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 08:42:24,568 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:42:24,569 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:42:24,657 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 08:42:24,657 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 519.6 ms 08:42:24,657 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 08:42:24,658 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:42:24,658 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:42:24,659 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:42:24,659 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:42:24,659 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:42:24,660 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:24,660 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:42:24,660 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:42:24,660 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@496c260 08:42:24,660 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@351d920f 08:42:24,660 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3eb22a3c 08:42:24,660 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@766a2217 08:42:24,660 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3eb22a3c 08:42:24,660 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@766a2217 08:42:24,660 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 08:42:24,660 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 08:42:24,967 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 08:42:24,967 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 307.2 ms 08:42:24,967 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 08:42:24,968 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:42:24,969 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:42:25,267 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 08:42:25,268 AM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 607.4 ms 08:42:25,268 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 08:42:25,268 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:42:25,269 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:42:25,269 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:42:25,269 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:42:25,270 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:25,270 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:25,273 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:42:25.275] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:25,291 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:42:25,303 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [9024544529785512990] 08:42:25,303 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:42:25,303 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:42:25,303 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:42:25,303 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:25,303 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:25,322 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:42:25,323 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:42:25,323 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:42:25,324 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:25,326 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:25,328 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 08:42:25,330 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-890239974] 08:42:25,331 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-890239974] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,331 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:42:25,332 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:42:25,332 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:42:25,332 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,332 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:25,332 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:25,332 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,332 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:25,338 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:42:25,338 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@28b0cde1} 08:42:25,338 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 08:42:25,339 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:25,341 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:25,343 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 08:42:25,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:25,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,344 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-1-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1170580231] 08:42:25,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:25,344 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:25,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:25,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-1-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#-959373791] 08:42:25,345 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 08:42:25,345 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1170580231] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,346 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1862379628 created and ready for shard:member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,346 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1862379628] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,346 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#2040015796] 08:42:25,347 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:42:25,347 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:42:25,348 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 08:42:25,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:42:25,348 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:42:25,349 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,349 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,349 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 08:42:25,349 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:25,349 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,349 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,349 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:25,350 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 08:42:25,350 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:42:25,351 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#2040015796] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,351 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#-959373791] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,351 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#1210074502] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,351 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#1210074502 created and ready for shard:member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:25,351 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@2a788cd0} 08:42:25,351 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 08:42:25,351 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:25,351 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,352 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:42:25,352 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,352 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,352 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-1-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#821091473] 08:42:25,352 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,352 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,352 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,352 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-1-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#2018010608] 08:42:25,352 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 08:42:25,353 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:42:25,353 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#821091473] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,353 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#347188689 created and ready for shard:member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,353 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#347188689] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,353 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:42:25,353 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardLeaderReplica is initialized 08:42:25,353 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#2018010608] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,353 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 08:42:25,354 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#1461068936 created and ready for shard:member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,354 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#1461068936] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,354 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:42:25,354 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,354 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,354 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 08:42:25,354 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 08:42:25,354 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 08:42:25,356 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 08:42:25,356 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 08:42:25,357 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 08:42:25,357 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:25,359 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:42:25,360 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,360 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,360 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:42:25,360 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is initialized 08:42:25,360 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 08:42:25,360 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 08:42:25,360 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:25,360 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 08:42:25,361 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:25,363 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:42:25,363 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,363 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,363 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:42:25,363 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardLeaderReplica is initialized 08:42:25,363 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 08:42:25,363 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 08:42:25,363 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 08:42:25,363 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:42:25,364 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,364 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,364 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:42:25,364 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardLeaderReplica is initialized 08:42:25,364 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 08:42:25,364 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 08:42:25,364 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from null to Follower 08:42:25,367 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:42:25,377 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-1449235344323093434] 08:42:25,377 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.1.5] ... 08:42:25,379 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:25,379 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:42:25,381 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:25,381 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:25,396 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:25,398 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:25,401 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 08:42:25,405 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1437000435] 08:42:25,405 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:42:25,405 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:42:25,405 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:42:25,405 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,406 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardLeaderReplica 08:42:25,407 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1437000435] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,413 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:42:25,415 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 08:42:25,415 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:25,416 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:25,417 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@5b6e584d} 08:42:25,417 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:25,417 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,418 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardLeaderReplica 08:42:25,420 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 08:42:25,420 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 08:42:25,420 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica#-1754833677] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,421 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:42:25,421 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-1133895993] 08:42:25,422 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:42:25,422 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 08:42:25,422 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:42:25,422 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:42:25,422 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,422 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,422 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 08:42:25,424 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#-1757728407] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,424 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1712006574 created and ready for shard:member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,425 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-1133895993] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,425 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-478724423] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,425 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1712006574] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,426 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:25,426 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:42:25,428 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 08:42:25,429 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:42:25,429 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:25,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:42:25,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardLeaderReplica is initialized 08:42:25,430 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,430 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,430 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 08:42:25,430 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#-1757728407 created and ready for shard:member-2-shard-people-config_testRemoveShardLeaderReplica 08:42:25,430 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 08:42:25,430 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 08:42:25,430 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardLeaderReplica from null to Follower 08:42:25,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@5e2149ae} 08:42:25,431 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 08:42:25,431 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:25,431 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,432 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,432 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 08:42:25,432 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica#-1065287661] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,433 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:42:25,433 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1290608938 created and ready for shard:member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,433 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:42:25,433 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1290608938] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,434 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 08:42:25,434 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica#-1925931530] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,434 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 08:42:25,435 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#1139434959 created and ready for shard:member-2-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,435 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:42:25,435 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:42:25,435 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardLeaderReplica is initialized 08:42:25,435 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#1139434959] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,436 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 08:42:25,437 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:25,439 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,439 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,439 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 08:42:25,439 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 08:42:25,439 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 08:42:25,439 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:42:25,440 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:42:25,440 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardLeaderReplica is initialized 08:42:25,440 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,440 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,440 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 08:42:25,440 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:25,440 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 08:42:25,440 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 08:42:25,444 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:42:25,446 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:42:25,447 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:42:25,447 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is initialized 08:42:25,447 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,447 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,447 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 08:42:25,448 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 08:42:25,448 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 08:42:25,452 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:42:25,457 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:42:25,457 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-1584252146076269381] 08:42:25,457 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.1.5] ... 08:42:25,458 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:25,458 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,458 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,458 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,458 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,458 AM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,458 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,459 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:25,459 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,459 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,459 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,459 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:25,460 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:42:25,459 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:25,460 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:42:25,460 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:25,460 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:25,460 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:25,470 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:25,475 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:25,477 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 08:42:25,480 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-2089724357] 08:42:25,481 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:42:25,481 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:42:25,481 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:42:25,481 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,481 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardLeaderReplica 08:42:25,488 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-2089724357] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,490 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:42:25,493 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:25,493 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,493 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:25,494 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:42:25,499 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:42:25,500 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@7a15a8e4} 08:42:25,500 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 08:42:25,500 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:25,500 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,500 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:25,500 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,500 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,501 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-3-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#1127830663] 08:42:25,501 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardLeaderReplica 08:42:25,501 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 08:42:25,501 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:42:25,502 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:25,502 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 08:42:25,503 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 08:42:25,503 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:42:25,504 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#-105359125 created and ready for shard:member-3-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,504 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#1127830663] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,504 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#1209591475] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,504 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#-105359125] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,504 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#1690482992 created and ready for shard:member-3-shard-people-config_testRemoveShardLeaderReplica 08:42:25,504 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 08:42:25,504 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 08:42:25,505 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#1690482992] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,505 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-2070654341] 08:42:25,505 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-2070654341] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,505 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:42:25,505 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:42:25,506 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:42:25,506 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,506 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,506 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:25,506 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,506 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,506 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:25,506 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:42:25,506 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-3-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#1209591475] 08:42:25,509 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:25,509 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:25,512 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:42:25,512 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:42:25,512 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardLeaderReplica is initialized 08:42:25,513 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,513 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,513 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 08:42:25,513 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:42:25,513 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,513 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,513 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 08:42:25,513 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 08:42:25,513 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:42:25,513 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:25,513 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardLeaderReplica is initialized 08:42:25,513 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 08:42:25,513 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 08:42:25,513 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@35ef53ed} 08:42:25,514 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 08:42:25,514 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:25,514 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,514 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:42:25,514 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:25,514 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 08:42:25,514 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardLeaderReplica from null to Follower 08:42:25,514 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,514 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,514 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-3-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#193013531] 08:42:25,514 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,515 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,515 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,515 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-3-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#1467598777] 08:42:25,515 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#193013531] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,515 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 08:42:25,515 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#1467598777] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,515 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 08:42:25,515 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#14977796 created and ready for shard:member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,515 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#14977796] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,515 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#289218228 created and ready for shard:member-3-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,515 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#289218228] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:25,516 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:42:25,516 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:42:25,516 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 08:42:25,516 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 08:42:25,520 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:25,520 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:25,522 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:42:25,522 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:42:25,522 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is initialized 08:42:25,522 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,522 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,523 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 08:42:25,523 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:42:25,523 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:25,523 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:42:25,523 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardLeaderReplica is initialized 08:42:25,523 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,523 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,523 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 08:42:25,523 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 08:42:25,523 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:25,523 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 08:42:25,523 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 08:42:25,523 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 08:42:25,682 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 08:42:25,682 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:25,682 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,682 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,682 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 08:42:25,682 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 08:42:25,761 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 08:42:25,762 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:25,762 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,762 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,762 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 08:42:25,762 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Follower to Candidate 08:42:25,771 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 08:42:25,771 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:25,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,771 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 08:42:25,771 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Follower to Candidate 08:42:25,772 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 08:42:25,772 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:25,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:25,772 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:25,772 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 08:42:25,772 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Follower to Candidate 08:42:26,12 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 08:42:26,112 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 08:42:26,133 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 08:42:26,171 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 08:42:26,320 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:42:26,320 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:42:26,321 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:26,321 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:26,321 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:26,321 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:26,321 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:42:26,322 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:26,322 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:42:26,322 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,322 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,322 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,321 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,322 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:26,326 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:26,326 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:26,326 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:26,326 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,326 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,342 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 08:42:26,343 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:26,343 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:26,349 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:42:26,349 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:26,349 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@41e0698a 08:42:26,349 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 08:42:26,350 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 08:42:26,350 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 08:42:26,350 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 08:42:26,350 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@b0f6c22 08:42:26,351 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 08:42:26,351 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3bc398d3 08:42:26,413 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:26,413 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:26,413 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,413 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,413 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,413 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,413 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,413 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:26,413 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,413 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,413 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,413 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,413 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,413 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:42:26,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:42:26,414 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,414 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:26,414 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,414 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,414 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:42:26,414 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:42:26,414 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,414 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:42:26,475 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:42:26,502 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 08:42:26,504 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:26,504 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:26,511 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:42:26,511 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:26,511 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 08:42:26,511 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 08:42:26,511 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6847dd 08:42:26,511 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Candidate to Leader 08:42:26,512 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 08:42:26,512 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7ec111cf 08:42:26,512 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 08:42:26,512 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7a9b2029 08:42:26,532 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 08:42:26,534 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:26,534 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:26,540 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:42:26,540 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:26,540 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 08:42:26,540 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 08:42:26,540 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@77743ae7 08:42:26,541 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Candidate to Leader 08:42:26,541 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 08:42:26,541 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7115ae61 08:42:26,541 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardLeaderReplica status sync done false 08:42:26,541 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@745cc10a 08:42:26,541 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 08:42:26,541 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 08:42:26,541 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done false 08:42:26,593 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 08:42:26,594 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:26,595 AM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:26,601 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:42:26,601 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:26,601 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 08:42:26,601 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@a098d17 08:42:26,601 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 08:42:26,601 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Candidate to Leader 08:42:26,601 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 08:42:26,601 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} is now ready 08:42:26,601 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-890239974] 08:42:26,601 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:26,601 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1170580231] 08:42:26,601 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1170580231] 08:42:26,602 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardLeaderReplica status sync done false 08:42:26,602 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardLeaderReplica status sync done false 08:42:26,602 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1009535f 08:42:26,602 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1437000435] 08:42:26,602 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 08:42:26,602 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:42:26,602 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@70634b59 08:42:26,602 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 08:42:26,602 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-478724423] 08:42:26,602 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-478724423] 08:42:26,603 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-2089724357] 08:42:26,603 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:42:26,603 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#1127830663] 08:42:26,603 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#1127830663] 08:42:26,604 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-1}, datastoreType Config 08:42:26,604 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-1}] 08:42:26,604 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:42:26,605 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1170580231, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@3dff830a] 08:42:26,605 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1170580231 for shard member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,606 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardLeaderReplica} for shard cars 08:42:26,606 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 08:42:26,606 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 08:42:26,606 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1437000435] 08:42:26,606 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:42:26,607 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-478724423] 08:42:26,607 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-478724423] 08:42:26,610 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:42:26,611 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1170580231] with 10000 ms timeout 08:42:26,611 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 08:42:26,611 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1 : Local Shard replica for shard cars has been removed 08:42:26,611 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: persisting the shard list [people] 08:42:26,611 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 08:42:26,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@206da572 08:42:26,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@206da572 08:42:26,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 08:42:26,613 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2c023496 08:42:26,613 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,615 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5e2d8fd1 08:42:26,615 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ff5c911 08:42:26,615 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 08:42:26,618 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 4 08:42:26,618 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 4 08:42:26,618 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:26,618 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:26,618 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 08:42:26,618 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3561cb31 08:42:26,618 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 08:42:26,619 AM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 4 in "RequestVote{term=4, candidateId=member-3-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3}" message is greater than follower's term 3 - updating term 08:42:26,623 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 08:42:26,623 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:26,623 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 08:42:26,623 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 08:42:26,623 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@16d83fdc 08:42:26,623 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 08:42:26,623 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 08:42:26,624 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 08:42:26,624 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@68d7d53c 08:42:26,624 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 08:42:26,625 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 08:42:26,664 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-478724423] 08:42:26,664 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1437000435] 08:42:26,664 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:42:26,664 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-478724423] 08:42:26,664 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-478724423] 08:42:26,665 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-2089724357] 08:42:26,665 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:42:26,665 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#1127830663] 08:42:26,665 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#1127830663] 08:42:26,665 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-890239974] 08:42:26,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:26,665 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Local shard cars not found - shards present: [people] 08:42:26,666 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:42:26,666 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,666 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardLeaderReplica 08:42:26,666 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 08:42:26,666 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@41ef89c 08:42:26,666 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@41ef89c 08:42:26,666 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 08:42:26,667 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@70ec881f 08:42:26,667 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 08:42:26,667 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardLeaderReplica: journal closed 08:42:26,668 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@f1b6aad 08:42:26,668 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 08:42:26,672 AM [cluster-test-shard-dispatcher-32] [INFO] Candidate - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 5 08:42:26,672 AM [cluster-test-shard-dispatcher-32] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 5 08:42:26,672 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:26,672 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:26,672 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 08:42:26,672 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1a4be312 08:42:26,672 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 08:42:26,673 AM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 5 in "RequestVote{term=5, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 08:42:26,677 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 5 08:42:26,677 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:26,677 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:26,677 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 08:42:26,677 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7005658 08:42:26,677 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 08:42:26,678 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 08:42:26,678 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:26,678 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 08:42:26,678 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 08:42:26,678 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5ec0ab47 08:42:26,678 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 08:42:26,678 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 08:42:26,679 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: leader changed to member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,679 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6e4d52c5 08:42:26,679 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to member-2-shard-cars-config_testRemoveShardLeaderReplica in 13.00 ms 08:42:26,679 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 08:42:26,679 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 08:42:26,680 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 08:42:26,680 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardLeaderReplica: journal closed 08:42:26,681 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:42:26,681 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:42:26,681 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 08:42:26,681 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:42:26,681 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardLeaderReplica 08:42:26,681 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:42:26,682 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 08:42:26,682 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 08:42:26,682 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardLeaderReplica: journal closed 08:42:26,682 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 08:42:26,683 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:42:26,683 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:42:26,683 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:26,684 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:26,687 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:26,687 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:26,687 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:26,687 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:26,688 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:42:26,696 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:42:26,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,696 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardLeaderReplica 08:42:26,696 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:42:26,697 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6ca219ed 08:42:26,697 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1043fe9e 08:42:26,697 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1043fe9e 08:42:26,697 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 08:42:26,697 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 08:42:26,697 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardLeaderReplica: journal closed 08:42:26,831 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done true 08:42:26,832 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:42:26,860 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done true 08:42:26,861 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:42:26,991 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cannot append entries because sender's term 3 is less than 5 08:42:26,992 AM [cluster-test-shard-dispatcher-15] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 5 in "AppendEntriesReply{term=5, success=false, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}" message is greater than leader's term 3 - switching to Follower 08:42:26,994 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 5 08:42:26,994 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:42:26,994 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:26,995 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 08:42:26,995 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@66a92286 08:42:26,995 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 08:42:26,995 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 08:42:27,152 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:27,181 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:27,340 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:42:27,471 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:27,501 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:27,699 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-1584252146076269381] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:42:27,704 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-1584252146076269381] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:42:27,791 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:27,821 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:28,111 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:28,141 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:28,431 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:28,461 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:28,629 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader not elected in time 08:42:28,629 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to null in 2.018 s 08:42:28,630 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 08:42:28,630 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardLeaderReplica: journal closed 08:42:28,631 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1170580231] 08:42:28,752 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:28,781 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:29,72 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:29,102 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:29,392 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:29,422 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:29,712 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:29,741 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:42:30,32 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:30,61 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:30,353 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:30,382 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:30,671 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:42:30,701 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:42:30,992 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:31,21 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:31,311 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:31,341 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:31,630 AM [cluster-test-shard-dispatcher-31] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:42:31,630 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:42:31,630 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:31,630 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 08:42:31,630 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:42:31,633 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:42:31,633 AM [cluster-test-shard-dispatcher-31] [INFO] IsolatedLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica: IsolatedLeader member-1-shard-cars-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 08:42:31,633 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 08:42:31,634 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:31,634 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 08:42:31,634 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 08:42:31,634 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 08:42:31,659 AM [cluster-test-shard-dispatcher-15] [WARN] Leader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:42:31,659 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:42:31,660 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:31,660 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 08:42:31,660 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:42:31,662 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:31,662 AM [cluster-test-shard-dispatcher-15] [INFO] IsolatedLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica: IsolatedLeader member-1-shard-people-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 08:42:31,662 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 08:42:31,662 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:31,662 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 08:42:31,662 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 08:42:31,662 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 08:42:31,720 AM [cluster-test-shard-dispatcher-31] [WARN] Leader - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:42:31,720 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:42:31,720 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:31,720 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 08:42:31,720 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:42:31,952 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 08:42:31,980 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:42:32,272 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:32,301 AM [cluster-test-shard-dispatcher-31] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:32,440 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 08:42:32,440 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:42:32,441 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:32,441 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:32,515 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 08:42:32,516 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:32,516 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:32,592 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:32,620 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:32,911 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:32,941 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:42:33,231 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:33,261 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:33,553 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 08:42:33,581 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:33,872 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:42:33,901 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:33,963 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:42:34,191 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:34,220 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:34,358 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:42:34,510 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:34,541 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:34,831 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:34,861 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:35,153 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:42:35,181 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:35,472 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:42:35,501 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:35,792 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:42:35,822 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:42:35,955 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer aborted 08:42:35,955 AM [cluster-test-shard-dispatcher-31] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: Failed to transfer leadership in 9.258 s 08:42:35,955 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In unpauseLeader 08:42:35,956 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 08:42:35,956 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardLeaderReplica: journal closed 08:42:35,957 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:42:35,957 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:42:35,957 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 08:42:35,958 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:42:35,958 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardLeaderReplica 08:42:35,958 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:42:35,958 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 08:42:35,958 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 08:42:35,959 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 08:42:35,959 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardLeaderReplica: journal closed 08:42:35,960 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:42:35,960 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:42:35,961 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:35,961 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:36,749 AM [cluster-test-shard-dispatcher-15] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:42:36,750 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:42:36,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:36,750 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 08:42:36,750 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:42:36,779 AM [cluster-test-shard-dispatcher-15] [WARN] Leader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:42:36,780 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:42:36,780 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:42:36,780 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 08:42:36,780 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:42:36,976 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:36,977 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:36,979 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:42:36,979 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-1449235344323093434] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [08/30/2025 08:42:36.981] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:36,985 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:42:36,986 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardLeaderReplica 08:42:36,986 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 08:42:36,986 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 08:42:36,987 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardLeaderReplica: journal closed 08:42:36,988 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:42:36,988 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:42:36,988 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 08:42:36,989 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:42:36,989 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardLeaderReplica 08:42:36,989 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:42:36,989 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 08:42:36,989 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 08:42:36,989 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 08:42:36,989 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardLeaderReplica: journal closed 08:42:36,989 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:42:36,989 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:42:36,991 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:36,991 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:36,993 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:42:36.994] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:37,12 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:42:37,20 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [4658977635552392530] 08:42:37,20 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:42:37,22 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:42:37,22 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:42:37,24 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:37,24 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:37,36 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,37 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,37 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:42:37,37 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:42:37,37 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:42:37,39 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,42 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-1193841250] 08:42:37,42 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:42:37,42 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:42:37,42 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:42:37,42 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,42 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,42 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:37,42 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,42 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,44 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-1193841250] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,50 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 08:42:37,50 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@94d904c} 08:42:37,51 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 08:42:37,51 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:37,51 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,52 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,53 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,56 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,56 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,56 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,56 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-1767500583] 08:42:37,56 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,58 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 08:42:37,58 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-1767500583] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,59 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard#-226995237] 08:42:37,59 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard#-226995237] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,59 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#2145609407] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,59 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#2145609407 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,59 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:42:37,59 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:42:37,59 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:42:37,59 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,60 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,60 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:37,60 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,60 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,60 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-1343297362] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,60 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 08:42:37,61 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1490431286 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,61 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1490431286] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,61 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 08:42:37,61 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 08:42:37,62 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 08:42:37,62 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 08:42:37,63 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,64 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,64 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-1343297362] 08:42:37,64 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,65 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,69 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 08:42:37,69 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@e051ec0} 08:42:37,69 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 08:42:37,70 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,70 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 08:42:37,70 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,70 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,70 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,70 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 08:42:37,70 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:42:37,70 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 08:42:37,70 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:42:37,70 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 08:42:37,71 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:37,71 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,71 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,71 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,71 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 08:42:37,71 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,71 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,71 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#163242312] 08:42:37,71 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,71 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,71 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,71 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#-1834534781] 08:42:37,72 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 08:42:37,72 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#163242312] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,72 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#-1834534781] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,72 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 08:42:37,72 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 08:42:37,72 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:37,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,73 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1717740901 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,73 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:42:37,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 08:42:37,73 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 08:42:37,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 08:42:37,73 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 08:42:37,73 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,73 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1697829354] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,73 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1717740901] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,73 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@46b7752e 08:42:37,73 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 08:42:37,75 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 08:42:37,75 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:37,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,75 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1697829354 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,75 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 08:42:37,75 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:42:37,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 08:42:37,75 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 08:42:37,75 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 08:42:37,75 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,75 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@786e6c06 08:42:37,75 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 08:42:37,75 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForSingleNodeShard is ready 08:42:37,75 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} is now ready 08:42:37,78 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard ShardName{value=cars}: {member-1=false} 08:42:37,78 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-1=false}] 08:42:37,78 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:37,79 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 08:42:37,80 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 08:42:37,81 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,82 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 08:42:37,83 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,83 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 08:42:37,83 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,83 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,84 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:42:37,84 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 08:42:37,84 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 08:42:37,85 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,87 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 08:42:37,87 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:37,87 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,87 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,87 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:42:37,87 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 08:42:37,87 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 08:42:37,87 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 08:42:37,87 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 08:42:37,87 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,87 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@365bfcf9 08:42:37,87 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 08:42:37,87 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,88 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,88 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,88 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:42:37,88 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 08:42:37,88 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 08:42:37,89 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,89 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=INVALID_REQUEST, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,90 AM [cluster-test-pekko.actor.default-dispatcher-23] [ERROR] ClusterAdminRpcService - Failed to change member voting states for shard ShardName{value=cars} java.lang.IllegalArgumentException: The requested voting state change for shard cars is invalid. At least one member must be voting at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$10.onComplete(ShardManager.java:1541) at org.apache.pekko.dispatch.OnComplete.internal(Future.scala:310) at org.apache.pekko.dispatch.OnComplete.internal(Future.scala:307) at org.apache.pekko.dispatch.japi$CallbackBridge.apply(Future.scala:234) at org.apache.pekko.dispatch.japi$CallbackBridge.apply(Future.scala:231) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at org.apache.pekko.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:73) at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:110) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:110) at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59) at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:61) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 08:42:37,90 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 08:42:37,90 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:37,90 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,90 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,90 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 08:42:37,90 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:42:37,90 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 08:42:37,90 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 08:42:37,90 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 08:42:37,90 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,90 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-1193841250] 08:42:37,90 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:37,91 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@12003996 08:42:37,91 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-1767500583] 08:42:37,91 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 08:42:37,91 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForSingleNodeShard is ready 08:42:37,91 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-1767500583] 08:42:37,91 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 08:42:37,92 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,92 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,92 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:42:37,92 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@3d6cfd6f 08:42:37,92 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@3d6cfd6f 08:42:37,92 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@4f926b69 08:42:37,92 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@4f926b69 08:42:37,92 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 08:42:37,92 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 08:42:37,92 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: journal closed 08:42:37,92 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: journal closed 08:42:37,93 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:42:37,93 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:42:37,93 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 08:42:37,93 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,93 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:42:37,93 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:42:37,93 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@496d5f6 08:42:37,93 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@496d5f6 08:42:37,93 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@41944b00 08:42:37,93 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 08:42:37,93 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@41944b00 08:42:37,93 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: journal closed 08:42:37,94 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 08:42:37,94 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: journal closed 08:42:37,94 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:42:37,94 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:42:37,95 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:37,95 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:37,96 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:42:37,109 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:42:37,123 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-7537711420680726500] 08:42:37,123 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.1.5] ... 08:42:37,130 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:42:37,130 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:42:37,133 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:37,133 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:37,164 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:42:37,164 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:42:37,165 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,166 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:42:37,167 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,171 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 08:42:37,190 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-389539634] 08:42:37,191 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-389539634] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,194 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:42:37,194 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:42:37,194 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:42:37,195 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,195 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,195 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:37,195 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,195 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,201 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:42:37,202 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@11c9c80c} 08:42:37,202 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 08:42:37,203 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:37,203 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,203 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,204 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,204 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,204 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#1210698480] 08:42:37,204 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,205 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,205 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,205 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard on shard member-1-shard-people-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#814549497] 08:42:37,205 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#1210698480] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,205 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:42:37,206 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,207 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-771984602 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,207 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:42:37,207 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#-771984602] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,208 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:42:37,209 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-535104849 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,210 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:42:37,211 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,213 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 08:42:37,213 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,214 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#814549497] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,214 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-535104849] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,214 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:42:37,216 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:42:37,216 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-1702177226] 08:42:37,216 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-1702177226] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,216 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:42:37,216 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 08:42:37,217 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:42:37,217 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:42:37,217 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:42:37,217 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,217 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,217 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,217 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 08:42:37,218 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 08:42:37,218 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:37,218 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,218 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,218 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@454e3fe1} 08:42:37,218 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:37,218 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,219 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:42:37,219 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:42:37,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is initialized 08:42:37,220 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:42:37,220 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:42:37,221 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:42:37,221 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 08:42:37,221 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#1899927341] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,221 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-950991410 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,221 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-950991410] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,222 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 08:42:37,222 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,222 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,222 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:42:37,222 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 08:42:37,223 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 08:42:37,226 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,226 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,226 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#1899927341] 08:42:37,226 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,227 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,227 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,227 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-699739239] 08:42:37,227 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:42:37,227 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-699739239] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,228 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1842486436 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,228 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:42:37,228 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,228 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1842486436] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,228 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:42:37,230 AM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,230 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,230 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:42:37,230 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 08:42:37,230 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,230 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,231 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:42:37,231 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 08:42:37,231 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 08:42:37,232 AM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,233 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:42:37,233 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 08:42:37,233 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,233 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,233 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:42:37,233 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 08:42:37,233 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 08:42:37,240 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:42:37,256 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-7388696750411403718] 08:42:37,256 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.1.5] ... 08:42:37,261 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:37,261 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:42:37,261 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:37,261 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:37,270 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,271 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,273 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 08:42:37,278 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#622523435] 08:42:37,278 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#622523435] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,279 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:42:37,279 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:42:37,279 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:42:37,279 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,279 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,284 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:42:37,285 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@42ec0781} 08:42:37,285 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 08:42:37,285 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,287 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,289 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 08:42:37,290 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:37,290 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,297 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,298 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:42:37,299 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:42:37,300 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#1763721540] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,300 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:42:37,300 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:42:37,300 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-235442085] 08:42:37,300 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:42:37,301 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:42:37,301 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:42:37,301 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,301 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-367471042 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,301 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,301 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:42:37,302 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#843367914] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,302 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#843367914 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,302 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:42:37,305 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,305 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,306 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard#228188929] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,306 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-235442085] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,306 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-367471042] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,308 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,308 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,308 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,308 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:42:37,309 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:42:37,309 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is initialized 08:42:37,309 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,309 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 08:42:37,309 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 08:42:37,309 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,309 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,309 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:42:37,309 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:42:37,309 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 08:42:37,309 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 08:42:37,309 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 08:42:37,309 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 08:42:37,309 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@733340c7} 08:42:37,310 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 08:42:37,310 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:37,310 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,310 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,311 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:42:37,311 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:42:37,312 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#-1076690732] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,312 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:42:37,312 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-158938291 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,312 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-158938291] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,312 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:42:37,312 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#530655201] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,312 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:42:37,313 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1002329978] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,313 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:42:37,317 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:42:37,319 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:42:37,321 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1002329978 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,321 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:37,321 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,321 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,323 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,323 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,326 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,326 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,326 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,326 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:42:37,327 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:37,328 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,328 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,328 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,328 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,328 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,328 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:42:37,328 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 08:42:37,328 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 08:42:37,328 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 08:42:37,332 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,332 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,332 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,333 AM [cluster-test-shard-dispatcher-35] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,335 AM [cluster-test-shard-dispatcher-35] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,335 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:42:37,335 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 08:42:37,335 AM [cluster-test-shard-dispatcher-35] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,335 AM [cluster-test-shard-dispatcher-35] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,335 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:42:37,335 AM [cluster-test-shard-dispatcher-35] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,336 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 08:42:37,336 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 08:42:37,336 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:42:37,346 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-5858669472599147070] 08:42:37,346 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.1.5] ... 08:42:37,349 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:37,349 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:42:37,350 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:42:37,350 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:42:37,363 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,366 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,370 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 08:42:37,373 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1584630491] 08:42:37,374 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:42:37,374 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:42:37,374 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:42:37,374 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,374 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,375 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1584630491] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,379 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:42:37,380 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@538d8231} 08:42:37,381 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 08:42:37,381 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:37,381 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,381 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,382 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:42:37,385 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 08:42:37,388 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,388 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#728580628] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,390 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:42:37,390 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:42:37,392 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:42:37,394 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:37,394 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,394 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,394 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,394 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,395 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:42:37,395 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard#-598184998] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,395 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1367109836] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,395 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1733612616 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,395 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:42:37,396 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:42:37,397 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-1883914339] 08:42:37,397 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:42:37,397 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:42:37,397 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:42:37,397 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,397 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,398 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:42:37,398 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,398 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,398 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 08:42:37,398 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f0d5474e218@40ec7247} 08:42:37,399 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 08:42:37,399 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:42:37,399 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,399 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:42:37,400 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,402 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,404 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:42:37,404 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-1744873318 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,404 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1367109836 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,404 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:42:37,405 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:42:37,405 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:42:37,405 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is initialized 08:42:37,406 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,406 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:42:37,407 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1733612616] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,407 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-1883914339] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,407 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#1507112106] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,407 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-1744873318] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,408 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:42:37,408 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,408 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,408 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#1507112106] 08:42:37,408 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,408 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,408 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,408 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,408 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#-1674958156] 08:42:37,408 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:42:37,408 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 08:42:37,409 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:42:37,409 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,409 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,409 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#-1674958156] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,409 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:42:37,409 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-479368935 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,409 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:42:37,409 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-479368935] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:42:37,410 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,410 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:42:37,410 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:37,410 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 08:42:37,411 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 08:42:37,411 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:42:37,411 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,411 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,411 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:42:37,411 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:37,411 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 08:42:37,411 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 08:42:37,412 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,413 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:42:37,413 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:37,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:42:37,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 08:42:37,413 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 08:42:37,413 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 08:42:37,413 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:42:37,413 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:42:37,413 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 08:42:37,413 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,413 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,413 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:37,414 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:42:37,414 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 08:42:37,414 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 08:42:37,591 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 08:42:37,591 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:37,591 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,591 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,591 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 08:42:37,591 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 08:42:37,600 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 08:42:37,600 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:37,600 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,600 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,600 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 08:42:37,600 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Follower to Candidate 08:42:37,621 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 08:42:37,621 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:37,621 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,621 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,621 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 08:42:37,621 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Follower to Candidate 08:42:37,640 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 08:42:37,640 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:42:37,640 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:42:37,640 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:42:37,640 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 08:42:37,640 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 08:42:37,981 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 08:42:37,990 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 08:42:37,990 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 08:42:38,50 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 08:42:38,149 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:38,149 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,149 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:42:38,149 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:38,149 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:42:38,149 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,149 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,149 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:38,149 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,149 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,149 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,149 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,150 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,150 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,150 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,150 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,150 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,150 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,150 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,150 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:38,150 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,150 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,151 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,151 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,151 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,151 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,156 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:38,156 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,156 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,156 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,156 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,156 AM [cluster-test-shard-dispatcher-35] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,156 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:38,157 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:38,157 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:38,157 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,157 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:42:38,286 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,286 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,311 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 08:42:38,313 AM [cluster-test-shard-dispatcher-35] [INFO] Follower - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:38,313 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:38,323 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:42:38,324 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 08:42:38,324 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 08:42:38,324 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@60ff1313 08:42:38,324 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 08:42:38,324 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 08:42:38,324 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7b38c55a 08:42:38,324 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 08:42:38,326 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 08:42:38,326 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@591003b1 08:42:38,329 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 08:42:38,330 AM [cluster-test-shard-dispatcher-35] [INFO] Follower - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:38,330 AM [cluster-test-shard-dispatcher-35] [INFO] Follower - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:38,336 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:42:38,336 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 08:42:38,336 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 08:42:38,336 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 08:42:38,336 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6e38e17c 08:42:38,336 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Candidate to Leader 08:42:38,337 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 08:42:38,337 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@787c80ce 08:42:38,337 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2f8f2088 08:42:38,337 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 08:42:38,364 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:42:38,379 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 08:42:38,381 AM [cluster-test-shard-dispatcher-35] [INFO] Follower - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:38,381 AM [cluster-test-shard-dispatcher-35] [INFO] Follower - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:38,388 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:42:38,388 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 08:42:38,388 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 08:42:38,388 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 08:42:38,389 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7081a1f0 08:42:38,389 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 08:42:38,389 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 08:42:38,389 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@22b927d2 08:42:38,389 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 08:42:38,389 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 08:42:38,390 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 08:42:38,390 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3d7e11a5 08:42:38,390 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 08:42:38,411 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 08:42:38,412 AM [cluster-test-shard-dispatcher-35] [INFO] Follower - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:38,413 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:42:38,419 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:42:38,419 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 08:42:38,420 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 08:42:38,420 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 08:42:38,420 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6a077863 08:42:38,420 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Candidate to Leader 08:42:38,420 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 08:42:38,420 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 08:42:38,420 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:42:38,421 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 08:42:38,421 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@68f2c4af 08:42:38,421 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 08:42:38,421 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@40357126 08:42:38,421 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 08:42:38,421 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 08:42:38,421 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 08:42:38,421 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-389539634] 08:42:38,421 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:38,421 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#1210698480] 08:42:38,422 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#1210698480] 08:42:38,422 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#622523435] 08:42:38,422 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:42:38,422 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#1763721540] 08:42:38,423 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#1763721540] 08:42:38,423 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1584630491] 08:42:38,423 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:42:38,423 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#728580628] 08:42:38,423 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#728580628] 08:42:38,424 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard ShardName{value=cars}: {member-2=false, member-3=false} 08:42:38,424 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 08:42:38,425 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:42:38,425 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,427 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]} has been successfully replicated to a majority of followers 08:42:38,428 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatusForShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,428 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 08:42:38,428 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully changed member voting states for shard ShardName{value=cars} 08:42:38,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-389539634] 08:42:38,430 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:42:38,430 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#1210698480] 08:42:38,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#1210698480] 08:42:38,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#622523435] 08:42:38,430 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:42:38,430 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#1763721540] 08:42:38,430 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#1763721540] 08:42:38,433 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 08:42:38,434 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1584630491] 08:42:38,434 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 08:42:38,434 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:42:38,434 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#728580628] 08:42:38,434 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#728580628] 08:42:38,434 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:42:38,435 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,435 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 08:42:38,435 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,435 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 08:42:38,436 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 08:42:38,436 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 08:42:38,436 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 08:42:38,436 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:42:38,436 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:42:38,436 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 08:42:38,436 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,436 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,437 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:42:38,437 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 08:42:38,437 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 08:42:38,437 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 08:42:38,437 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 08:42:38,437 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:42:38,437 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:42:38,438 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:38,438 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:38,443 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:38,443 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:38,443 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:38,443 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:38,445 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:42:38,455 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:42:38,455 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:38,455 AM [cluster-test-shard-dispatcher-35] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:42:38,455 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:42:38,455 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 08:42:38,456 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 08:42:38,457 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 08:42:38,457 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 08:42:38,457 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:42:38,458 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:42:38,458 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 08:42:38,458 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:38,458 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:38,458 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:42:38,458 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 08:42:38,459 AM [cluster-test-shard-dispatcher-35] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 08:42:38,459 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 08:42:38,459 AM [cluster-test-shard-dispatcher-35] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 08:42:38,459 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:42:38,459 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:42:38,462 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:38,462 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:39,168 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:42:39,457 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-5858669472599147070] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:42:39,475 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:39,475 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:42:39,477 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-7388696750411403718] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:42:39,478 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:42:39.480] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:42:39,482 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:42:39,482 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:42:39,482 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:42:39,482 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:42:39,483 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@790f4f6f 08:42:39,483 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@790f4f6f 08:42:39,483 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 08:42:39,483 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2f05eb47 08:42:39,483 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2f05eb47 08:42:39,483 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 08:42:39,483 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@37fa123f 08:42:39,483 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1ee026bd 08:42:39,927 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer aborted 08:42:39,927 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 444.3 ms 08:42:39,927 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In unpauseLeader 08:42:39,928 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 08:42:39,928 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 08:42:40,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer aborted 08:42:40,17 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 534.3 ms 08:42:40,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In unpauseLeader 08:42:40,18 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 08:42:40,18 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 08:42:40,19 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:42:40,19 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:42:40,19 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 08:42:40,20 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:42:40,20 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:42:40,20 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:42:40,20 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@55acdecb 08:42:40,20 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4ff58162 08:42:40,20 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4ff58162 08:42:40,20 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e26d30 08:42:40,20 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 08:42:40,20 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@9da828c 08:42:40,20 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@9da828c 08:42:40,20 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 08:42:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 08:42:40,557 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 536.8 ms 08:42:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 08:42:40,558 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 08:42:40,558 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 08:42:40,628 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 08:42:40,628 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 607.7 ms 08:42:40,628 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 08:42:40,629 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 08:42:40,629 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 08:42:40,630 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:42:40,630 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:42:40,631 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:42:40,631 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:42:40,633 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [08/30/2025 08:42:40.635] [cluster-test-pekko.actor.internal-dispatcher-5] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config