09:09:39,817 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:09:40,479 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [4998110356378244409] 09:09:40,513 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:09:40,703 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:09:40,703 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:09:40,761 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:09:40,761 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:09:41,139 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:09:41,148 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:09:41,172 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:41,176 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:09:41,188 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:42,266 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testBackupDatastore 09:09:42,345 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:42,387 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:09:42,387 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:09:42,388 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:09:42,390 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:42,392 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testBackupDatastore 09:09:42,472 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testBackupDatastore 09:09:42,496 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:42,499 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 09:09:42,499 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 09:09:42,499 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@5dc8fbe4} 09:09:42,504 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} started 09:09:42,510 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:42,511 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testBackupDatastore 09:09:42,512 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 09:09:42,513 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 09:09:42,514 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore on shard member-1-shard-cars-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#1848268457] 09:09:42,514 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testBackupDatastore 09:09:42,515 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 09:09:42,515 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 09:09:42,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore on shard member-1-shard-people-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-1693176241] 09:09:42,516 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-1693176241] 09:09:42,518 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-1693176241] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:42,519 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#1848268457] 09:09:42,520 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#1848268457] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:42,556 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Shard created, persistent : true 09:09:42,557 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Shard created, persistent : true 09:09:42,562 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:42,563 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:42,564 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 09:09:42,615 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:42,616 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:42,616 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 09:09:42,666 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:42,667 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:42,668 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 09:09:42,676 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#-1129985781 created and ready for shard:member-1-shard-people-config_testBackupDatastore 09:09:42,677 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#-1129985781] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:42,680 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testBackupDatastore: Starting recovery with journal batch size 1 09:09:42,685 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#-373706672 created and ready for shard:member-1-shard-cars-config_testBackupDatastore 09:09:42,686 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#-373706672] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:42,686 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-config_testBackupDatastore: Starting recovery with journal batch size 1 09:09:42,718 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:42,719 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:42,719 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 09:09:42,770 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:42,770 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:42,770 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 09:09:42,821 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:42,822 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:42,822 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 09:09:42,874 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:42,875 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:42,875 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 09:09:42,926 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:42,927 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:42,927 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 09:09:42,935 AM [cluster-test-shard-dispatcher-13] [INFO] FileMapper - Using io.netty.util.internal.PlatformDependent for ByteBuffer cleanup 09:09:42,949 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testBackupDatastore: journal open: applyTo=0 09:09:42,949 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testBackupDatastore: journal open: applyTo=0 09:09:42,978 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:42,979 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:42,979 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 09:09:42,980 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 09:09:42,990 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-config_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 09:09:42,993 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testBackupDatastore: Recovery completed in in 16.01 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:42,993 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-config_testBackupDatastore: Recovery completed in in 16.01 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:42,999 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:09:42,999 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testBackupDatastore is initialized 09:09:43,6 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:09:43,7 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:09:43,7 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testBackupDatastore is initialized 09:09:43,7 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:09:43,7 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:43,7 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:43,10 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from null to Follower 09:09:43,10 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from null to Follower 09:09:43,33 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-config_testBackupDatastore (Candidate): Starting new election term 1 09:09:43,33 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testBackupDatastore (Candidate): Starting new election term 1 09:09:43,33 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:43,33 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:43,34 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:43,34 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#1848268457] 09:09:43,33 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:43,33 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:09:43,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:09:43,35 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:43,38 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Follower to Candidate 09:09:43,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:43,38 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:43,39 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 09:09:43,39 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:43,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 09:09:43,39 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Follower to Candidate 09:09:43,45 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: became leader with frontend state for [] 09:09:43,45 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Candidate to Leader 09:09:43,48 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: became leader with frontend state for [] 09:09:43,49 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Candidate to Leader 09:09:43,57 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 09:09:43,58 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 09:09:43,58 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@55973e99 09:09:43,59 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testBackupDatastore from Candidate to Leader 09:09:43,59 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6299ae6b 09:09:43,59 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testBackupDatastore from Candidate to Leader 09:09:43,59 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testBackupDatastore is ready 09:09:43,109 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:43,110 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:09:43,110 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-1693176241] 09:09:43,111 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:43,119 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:43,126 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testBackupDatastore 09:09:43,128 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-923446244] 09:09:43,129 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-923446244] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:43,129 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:09:43,129 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:09:43,129 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:09:43,129 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testBackupDatastore 09:09:43,130 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testBackupDatastore 09:09:43,130 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:43,130 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 09:09:43,130 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 09:09:43,132 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@3248a068} 09:09:43,132 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} started 09:09:43,133 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:43,133 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testBackupDatastore 09:09:43,134 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#1335519227] 09:09:43,135 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Shard created, persistent : true 09:09:43,136 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-oper_testBackupDatastore: Starting recovery with journal batch size 1 09:09:43,136 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#1335519227] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:43,138 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testBackupDatastore: journal open: applyTo=0 09:09:43,139 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#-795889308] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:43,139 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 09:09:43,139 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 09:09:43,139 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore on shard member-1-shard-cars-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#1335519227] 09:09:43,139 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testBackupDatastore 09:09:43,140 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-1147018319] 09:09:43,141 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Shard created, persistent : true 09:09:43,142 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#1696814350 created and ready for shard:member-1-shard-people-oper_testBackupDatastore 09:09:43,142 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-people-oper_testBackupDatastore: Starting recovery with journal batch size 1 09:09:43,144 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testBackupDatastore: journal open: applyTo=0 09:09:43,145 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 09:09:43,145 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 09:09:43,145 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore on shard member-1-shard-people-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-1147018319] 09:09:43,145 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#-795889308 created and ready for shard:member-1-shard-cars-oper_testBackupDatastore 09:09:43,146 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-1147018319] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:43,147 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-oper_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 09:09:43,147 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-oper_testBackupDatastore: Recovery completed in in 9.233 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:43,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:09:43,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:43,149 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-people-oper_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 09:09:43,150 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-people-oper_testBackupDatastore: Recovery completed in in 5.611 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:43,150 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:09:43,150 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:43,151 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from null to Follower 09:09:43,152 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:09:43,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testBackupDatastore is initialized 09:09:43,152 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from null to Follower 09:09:43,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:09:43,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testBackupDatastore is initialized 09:09:43,154 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-people-oper_testBackupDatastore (Candidate): Starting new election term 1 09:09:43,155 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:43,155 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:09:43,155 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:43,155 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:43,155 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 09:09:43,155 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: became leader with frontend state for [] 09:09:43,158 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Follower to Candidate 09:09:43,158 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Candidate to Leader 09:09:43,158 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 09:09:43,158 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#1696814350] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:43,159 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-oper_testBackupDatastore (Candidate): Starting new election term 1 09:09:43,159 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:43,159 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 09:09:43,159 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:43,160 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:43,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 09:09:43,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: became leader with frontend state for [] 09:09:43,160 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@51e8bb33 09:09:43,160 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testBackupDatastore from Candidate to Leader 09:09:43,160 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Follower to Candidate 09:09:43,161 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Candidate to Leader 09:09:43,161 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 09:09:43,161 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@664bb96a 09:09:43,161 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testBackupDatastore from Candidate to Leader 09:09:43,161 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testBackupDatastore is ready 09:09:43,187 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-923446244] 09:09:43,188 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:09:43,188 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#1335519227] 09:09:43,239 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#-923446244] 09:09:43,239 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:09:43,240 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-1147018319] 09:09:43,282 AM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 09:09:43,285 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 09:09:43,286 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 09:09:43,293 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 09:09:43,305 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 09:09:43,312 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@20be74f1, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 09:09:43,316 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-cars-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@4089f122, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testBackupDatastore, ServerConfigPayload=null]] 09:09:43,322 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2058fdc3, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 09:09:43,322 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@933a702, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 09:09:43,323 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: All shard snapshots received 09:09:43,323 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 09:09:43,385 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClusterAdminRpcService - Successfully backed up datastore to file target/testBackupDatastore 09:09:43,419 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-1813460356] 09:09:43,419 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Received unhandled message org.opendaylight.controller.cluster.datastore.DatastoreContext@386805e0 09:09:43,421 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:43,421 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#1848268457] 09:09:43,461 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Stopping Shard 09:09:43,466 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testBackupDatastore: journal closed 09:09:43,474 AM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 09:09:43,475 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 09:09:43,478 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 09:09:43,479 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1a940f3e, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 09:09:43,479 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 09:09:43,480 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 09:09:43,481 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@727d5489, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 09:09:43,481 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@60833b00, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 09:09:43,481 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 09:09:43,695 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond 09:09:43,699 AM [cluster-test-pekko.actor.default-dispatcher-10] [ERROR] ClusterAdminRpcService - Failed to back up datastore to file target/testBackupDatastore java.util.concurrent.TimeoutException: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerGetSnapshotReplyActor.onReceive(ShardManagerGetSnapshotReplyActor.java:66) at org.apache.pekko.actor.UntypedAbstractActor$$anon$1.applyOrElse(AbstractActor.scala:341) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:481) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:273) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:234) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:246) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:09:43,712 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 09:09:43,718 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testBackupDatastore 09:09:43,721 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testBackupDatastore 09:09:43,725 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@18a4c952 09:09:43,726 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:09:43,727 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-config_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@18a4c952 09:09:43,729 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Stopping Shard 09:09:43,729 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-config_testBackupDatastore: journal closed 09:09:43,733 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:09:43,734 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:09:43,743 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 09:09:43,744 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testBackupDatastore 09:09:43,745 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testBackupDatastore 09:09:43,745 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5358e707 09:09:43,745 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5358e707 09:09:43,746 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@292842e2 09:09:43,746 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Stopping Shard 09:09:43,746 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@292842e2 09:09:43,746 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:09:43,746 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testBackupDatastore: journal closed 09:09:43,747 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Stopping Shard 09:09:43,748 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testBackupDatastore: journal closed 09:09:43,748 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:09:43,748 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:09:43,761 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:09:43,761 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:09:43,792 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:09:43,861 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:09:43,895 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-5076231917973634735] 09:09:43,896 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:09:43,902 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:09:43,903 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:09:43,918 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:09:43,919 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:09:43,936 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:43,937 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:09:43,938 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:09:43,938 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:09:43,940 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:43,944 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 09:09:43,947 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#573721712] 09:09:43,948 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:09:43,949 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:09:43,949 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:09:43,949 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddReplicasForAllShards 09:09:43,949 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testAddReplicasForAllShards 09:09:43,950 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:43,950 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 09:09:43,950 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 09:09:43,952 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@eb45cf5} 09:09:43,952 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 09:09:43,952 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#573721712] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:43,952 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:43,952 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddReplicasForAllShards 09:09:43,955 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 09:09:43,955 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 09:09:43,955 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards on shard member-1-shard-cars-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1462039311] 09:09:43,956 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testAddReplicasForAllShards 09:09:43,958 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1462039311] 09:09:43,959 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Shard created, persistent : true 09:09:43,960 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1462039311] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:43,961 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 09:09:43,961 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#225757823] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:43,961 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#225757823 created and ready for shard:member-1-shard-cars-config_testAddReplicasForAllShards 09:09:43,963 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddReplicasForAllShards: journal open: applyTo=0 09:09:43,963 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 09:09:43,963 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 09:09:43,963 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards on shard member-1-shard-people-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#909723195] 09:09:43,964 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#909723195] 09:09:43,964 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Shard created, persistent : true 09:09:43,965 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-people-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 09:09:43,967 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-people-config_testAddReplicasForAllShards: journal open: applyTo=0 09:09:43,968 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#1611553937 created and ready for shard:member-1-shard-people-config_testAddReplicasForAllShards 09:09:43,968 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#909723195] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:43,969 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#1611553937] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:43,969 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:09:43,969 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-1-shard-people-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:09:43,970 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-1-shard-people-config_testAddReplicasForAllShards: Recovery completed in in 2.789 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:43,970 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:43,970 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:43,971 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 09:09:43,972 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:09:43,972 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testAddReplicasForAllShards is initialized 09:09:43,972 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-config_testAddReplicasForAllShards: Recovery completed in in 9.009 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:43,973 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:09:43,973 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddReplicasForAllShards is initialized 09:09:43,973 AM [cluster-test-shard-dispatcher-10] [INFO] Candidate - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 09:09:43,973 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:43,974 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:43,974 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:43,974 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:43,974 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:09:43,974 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: became leader with frontend state for [] 09:09:43,975 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Follower to Candidate 09:09:43,975 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Candidate to Leader 09:09:43,975 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 09:09:43,976 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5f688d30 09:09:43,976 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Candidate to Leader 09:09:43,976 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:43,977 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:43,977 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 09:09:43,979 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 09:09:43,979 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:43,979 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:43,979 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:43,979 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:43,980 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:09:43,980 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: became leader with frontend state for [] 09:09:43,980 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Follower to Candidate 09:09:43,980 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Candidate to Leader 09:09:43,980 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 09:09:43,981 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4d2b31ac 09:09:43,981 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Candidate to Leader 09:09:43,981 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:09:44,2 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#573721712] 09:09:44,3 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:44,3 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1462039311] 09:09:44,54 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#573721712] 09:09:44,55 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:09:44,55 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#909723195] 09:09:44,56 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:44,61 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:44,66 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 09:09:44,69 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1008978226] 09:09:44,70 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1008978226] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:44,71 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:09:44,71 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:09:44,71 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:09:44,71 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddReplicasForAllShards 09:09:44,72 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testAddReplicasForAllShards 09:09:44,72 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@769ad7f7} 09:09:44,72 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:44,72 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 09:09:44,72 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 09:09:44,72 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 09:09:44,73 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:44,73 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddReplicasForAllShards 09:09:44,74 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#1549188700] 09:09:44,74 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Shard created, persistent : true 09:09:44,75 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-cars-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 09:09:44,76 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddReplicasForAllShards: journal open: applyTo=0 09:09:44,77 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#1549188700] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:44,77 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 09:09:44,77 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 09:09:44,77 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#1192492595 created and ready for shard:member-1-shard-cars-oper_testAddReplicasForAllShards 09:09:44,77 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards on shard member-1-shard-cars-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#1549188700] 09:09:44,77 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#1192492595] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:44,77 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testAddReplicasForAllShards 09:09:44,78 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 09:09:44,78 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 09:09:44,78 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#784407353] 09:09:44,78 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards on shard member-1-shard-people-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#784407353] 09:09:44,78 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-1-shard-cars-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:09:44,78 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Shard created, persistent : true 09:09:44,79 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-1-shard-cars-oper_testAddReplicasForAllShards: Recovery completed in in 2.663 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:44,79 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:44,79 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:44,79 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-people-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 09:09:44,80 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 09:09:44,80 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#784407353] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:44,80 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#-600439553 created and ready for shard:member-1-shard-people-oper_testAddReplicasForAllShards 09:09:44,80 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:09:44,80 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddReplicasForAllShards is initialized 09:09:44,81 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#-600439553] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:44,81 AM [cluster-test-shard-dispatcher-10] [INFO] Candidate - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 09:09:44,81 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:44,81 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:44,81 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:44,81 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testAddReplicasForAllShards: journal open: applyTo=0 09:09:44,81 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:44,82 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:09:44,82 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: became leader with frontend state for [] 09:09:44,82 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 09:09:44,82 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 09:09:44,82 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 09:09:44,83 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3d1c5496 09:09:44,83 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Candidate to Leader 09:09:44,83 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-people-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:09:44,84 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-people-oper_testAddReplicasForAllShards: Recovery completed in in 2.329 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:44,84 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:09:44,84 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testAddReplicasForAllShards is initialized 09:09:44,84 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:44,84 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:44,86 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 09:09:44,86 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:44,86 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:44,86 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:44,86 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:44,86 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:09:44,87 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: became leader with frontend state for [] 09:09:44,89 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 09:09:44,90 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 09:09:44,90 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 09:09:44,90 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 09:09:44,90 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@64fbe4fa 09:09:44,90 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Candidate to Leader 09:09:44,90 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 09:09:44,123 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1008978226] 09:09:44,124 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:09:44,124 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#1549188700] 09:09:44,174 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1008978226] 09:09:44,175 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:09:44,175 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#784407353] 09:09:44,181 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@136f2fc5] 09:09:44,186 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 09:09:44,187 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-528388922] 09:09:44,188 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Shard created, persistent : true 09:09:44,189 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-pets-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 09:09:44,189 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards-notifier#1027568396 created and ready for shard:member-1-shard-pets-config_testAddReplicasForAllShards 09:09:44,191 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testAddReplicasForAllShards: journal open: applyTo=0 09:09:44,193 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-1-shard-pets-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:09:44,194 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-1-shard-pets-config_testAddReplicasForAllShards: Recovery completed in in 2.853 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:44,194 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 09:09:44,194 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:44,194 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testAddReplicasForAllShards is initialized 09:09:44,194 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:44,194 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 09:09:44,197 AM [cluster-test-shard-dispatcher-10] [INFO] Candidate - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 09:09:44,197 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:44,197 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:44,197 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:44,197 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Follower to Candidate 09:09:44,198 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:44,198 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:09:44,198 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: became leader with frontend state for [] 09:09:44,198 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Candidate to Leader 09:09:44,198 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 09:09:44,198 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@52b258ea 09:09:44,198 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Candidate to Leader 09:09:44,199 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:09:44,240 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#573721712] 09:09:44,241 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 09:09:44,241 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-528388922] 09:09:44,269 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:09:44,296 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [5223643864626453675] 09:09:44,297 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.4.0] ... 09:09:44,305 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:09:44,305 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:09:44,308 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:09:44,308 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:09:44,344 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:44,347 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:44,360 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 09:09:44,370 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2080736686] 09:09:44,371 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 09:09:44,371 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2080736686] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:44,381 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:09:44,381 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 09:09:44,381 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 09:09:44,382 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@8c44326} 09:09:44,382 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:44,382 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:44,385 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:44,391 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 09:09:44,398 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#382209312] 09:09:44,399 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:09:44,399 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 09:09:44,400 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 09:09:44,401 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@603bc4e} 09:09:44,402 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#382209312] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:44,406 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 09:09:44,407 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:44,756 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:09:44,815 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:09:44,816 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:44,817 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:44,817 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:09:44,817 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 09:09:44,943 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:09:44,944 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:09:44,944 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:09:44,944 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards 09:09:44,944 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 09:09:44,944 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards 09:09:44,944 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards 09:09:44,944 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 09:09:44,944 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 09:09:44,944 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 09:09:44,947 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards 09:09:44,947 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 09:09:44,947 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards 09:09:44,947 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 09:09:44,947 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:09:45,21 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@23c2856] 09:09:45,23 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 09:09:45,24 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-633032912] 09:09:45,25 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Shard created, persistent : true 09:09:45,25 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-633032912] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:45,25 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=no-leader-ns, moduleName=no-leader-module, shardName=no-leader, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@76b330fc] 09:09:45,26 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2 doCreateShard: shardId: member-2-shard-no-leader-oper_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 09:09:45,27 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-pets-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 09:09:45,27 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#-1793503528 created and ready for shard:member-2-shard-pets-config_testAddReplicasForAllShards 09:09:45,27 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#-1793503528] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:45,28 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards#-90248869] 09:09:45,29 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards#-90248869] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:45,29 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Shard created, persistent : true 09:09:45,30 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#1317456011 created and ready for shard:member-2-shard-no-leader-oper_testAddReplicasForAllShards 09:09:45,30 AM [main] [INFO] ClusterAdminRpcService - Adding replicas for all shards 09:09:45,30 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#1317456011] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:45,30 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 09:09:45,30 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testAddReplicasForAllShards 09:09:45,31 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 09:09:45,32 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-no-leader-oper_testAddReplicasForAllShards: journal open: applyTo=0 09:09:45,32 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, no-leader] for data store oper_testAddReplicasForAllShards 09:09:45,33 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testAddReplicasForAllShards: journal open: applyTo=0 09:09:45,35 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 09:09:45,37 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 09:09:45,37 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=pets] 09:09:45,40 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-pets-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:09:45,38 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:09:45,41 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-pets-config_testAddReplicasForAllShards: Recovery completed in in 7.915 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:45,41 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 09:09:45,42 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:45,42 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:45,42 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 09:09:45,42 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=no-leader] 09:09:45,42 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:09:45,42 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 09:09:45,43 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:09:45,44 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Recovery completed in in 12.29 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:45,45 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:45,45 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:45,45 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , received role change from null to Follower 09:09:45,42 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 09:09:45,46 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:09:45,46 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 09:09:45,46 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=no-leader, waitUntilReady=true] 09:09:45,46 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for no-leader forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 09:09:45,47 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [no-leader] 09:09:45,47 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is initialized 09:09:45,47 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 09:09:45,47 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-no-leader-oper_testAddReplicasForAllShards from null to Follower 09:09:45,51 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:09:45,53 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 09:09:45,53 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 09:09:45,53 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for pets forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 09:09:45,53 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 09:09:45,53 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testAddReplicasForAllShards is initialized 09:09:45,55 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 09:09:45,56 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testAddReplicasForAllShards from null to Follower 09:09:45,59 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:09:45,60 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:09:45,61 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1462039311, primaryVersion=13] 09:09:45,61 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#1549188700, primaryVersion=13] 09:09:45,61 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:09:45,61 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:09:45,61 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#909723195, primaryVersion=13] 09:09:45,61 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#784407353, primaryVersion=13] 09:09:45,61 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 09:09:45,61 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-528388922, primaryVersion=13] 09:09:45,61 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=no-leader, waitUntilReady=true] 09:09:45,62 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: No shard found for no-leader 09:09:45,73 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: Failed to find leader for shard no-leader: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1003) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleReceive(ShardManager.java:240) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:37) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:37) at scala.PartialFunction.applyOrElse(PartialFunction.scala:225) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:224) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:33) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:280) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:281) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:481) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:273) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:234) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:246) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:09:45,73 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] ClusterAdminRpcService - Failed to add replica for shard ShardName{value=no-leader}, type Operational org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1003) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleReceive(ShardManager.java:240) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:37) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:37) at scala.PartialFunction.applyOrElse(PartialFunction.scala:225) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:224) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:33) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:280) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:281) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:481) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:273) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:234) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:246) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:09:45,73 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-528388922 for shard member-2-shard-pets-config_testAddReplicasForAllShards 09:09:45,75 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#1549188700 for shard member-2-shard-cars-oper_testAddReplicasForAllShards 09:09:45,78 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1462039311 for shard member-2-shard-cars-config_testAddReplicasForAllShards 09:09:45,78 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#-1558583254] 09:09:45,78 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-208615085] 09:09:45,79 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Shard created, persistent : true 09:09:45,79 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Shard created, persistent : true 09:09:45,80 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#-1558583254] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:45,80 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#1824915260 created and ready for shard:member-2-shard-cars-config_testAddReplicasForAllShards 09:09:45,80 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#1142828379 created and ready for shard:member-2-shard-cars-oper_testAddReplicasForAllShards 09:09:45,81 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 1 09:09:45,85 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-208615085] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:45,86 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-cars-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 09:09:45,86 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#1824915260] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:45,86 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#1142828379] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:45,87 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddReplicasForAllShards: journal open: applyTo=0 09:09:45,89 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-cars-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:09:45,90 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-cars-config_testAddReplicasForAllShards: Recovery completed in in 2.816 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:45,90 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:45,90 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:45,91 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#857334234] 09:09:45,91 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Shard created, persistent : true 09:09:45,92 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-people-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 09:09:45,93 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-notifier#-1594062678 created and ready for shard:member-2-shard-people-oper_testAddReplicasForAllShards 09:09:45,94 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 09:09:45,93 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testAddReplicasForAllShards: journal open: applyTo=0 09:09:45,90 AM [cluster-test-shard-dispatcher-13] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddReplicasForAllShards 09:09:45,90 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddReplicasForAllShards 09:09:45,90 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-pets-config_testAddReplicasForAllShards 09:09:45,96 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-people-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:09:45,97 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-people-oper_testAddReplicasForAllShards: Recovery completed in in 2.975 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:45,97 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:45,98 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:45,98 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 09:09:45,98 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#1562414916] 09:09:45,98 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 09:09:45,99 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Shard created, persistent : true 09:09:45,100 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-people-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 09:09:45,100 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-notifier#-1780735394 created and ready for shard:member-2-shard-people-config_testAddReplicasForAllShards 09:09:45,102 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddReplicasForAllShards: journal open: applyTo=0 09:09:45,103 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-people-config_testAddReplicasForAllShards: journal open: applyTo=0 09:09:45,104 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#909723195 for shard member-2-shard-people-config_testAddReplicasForAllShards 09:09:45,105 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-2-shard-cars-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:09:45,105 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:09:45,105 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddReplicasForAllShards is initialized 09:09:45,105 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-2-shard-cars-oper_testAddReplicasForAllShards: Recovery completed in in 2.794 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:45,106 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 09:09:45,106 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:45,106 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:45,106 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddReplicasForAllShards from null to Follower 09:09:45,106 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 09:09:45,106 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-people-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:09:45,108 AM [cluster-test-shard-dispatcher-13] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-config_testAddReplicasForAllShards 09:09:45,109 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-people-config_testAddReplicasForAllShards: Recovery completed in in 6.364 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:45,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:09:45,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:45,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:09:45,110 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testAddReplicasForAllShards is initialized 09:09:45,110 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 09:09:45,110 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 09:09:45,111 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testAddReplicasForAllShards from null to Follower 09:09:45,114 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#784407353 for shard member-2-shard-people-oper_testAddReplicasForAllShards 09:09:45,115 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:09:45,115 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testAddReplicasForAllShards is initialized 09:09:45,116 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:09:45,116 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddReplicasForAllShards is initialized 09:09:45,116 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 09:09:45,116 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 09:09:45,116 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testAddReplicasForAllShards from null to Follower 09:09:45,116 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddReplicasForAllShards from null to Follower 09:09:45,118 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-oper_testAddReplicasForAllShards 09:09:45,129 AM [cluster-test-shard-dispatcher-10] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 09:09:45,128 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 09:09:45,128 AM [cluster-test-shard-dispatcher-13] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 09:09:45,131 AM [cluster-test-shard-dispatcher-13] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:09:45,129 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 09:09:45,131 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:09:45,131 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:09:45,132 AM [cluster-test-shard-dispatcher-10] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:09:45,135 AM [cluster-test-shard-dispatcher-13] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 09:09:45,135 AM [cluster-test-shard-dispatcher-13] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:09:45,147 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 09:09:45,148 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 09:09:45,148 AM [cluster-test-shard-dispatcher-24] [INFO] Follower - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 09:09:45,149 AM [cluster-test-shard-dispatcher-13] [INFO] Follower - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 09:09:45,150 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 09:09:45,155 AM [cluster-test-shard-dispatcher-24] [INFO] Follower - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddReplicasForAllShards 09:09:45,155 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-people-config_testAddReplicasForAllShards 09:09:45,156 AM [cluster-test-shard-dispatcher-24] [INFO] SnapshotManager - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:09:45,156 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:09:45,157 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 09:09:45,157 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 09:09:45,157 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@12c0eeab 09:09:45,157 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@496fb08c 09:09:45,158 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddReplicasForAllShards 09:09:45,159 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:09:45,159 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5289ac2d 09:09:45,159 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 09:09:45,160 AM [cluster-test-shard-dispatcher-13] [INFO] Follower - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-pets-config_testAddReplicasForAllShards 09:09:45,160 AM [cluster-test-shard-dispatcher-13] [INFO] SnapshotManager - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:09:45,160 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 09:09:45,160 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@63f0a4a3 09:09:45,161 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:09:45,161 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-people-oper_testAddReplicasForAllShards 09:09:45,161 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:09:45,161 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1ae8844e 09:09:45,162 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 09:09:45,173 AM [cluster-test-shard-dispatcher-26] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: snapshot is durable as of 2026-03-18T09:09:45.131087870Z 09:09:45,173 AM [cluster-test-shard-dispatcher-13] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: snapshot is durable as of 2026-03-18T09:09:45.131100320Z 09:09:45,174 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: snapshot is durable as of 2026-03-18T09:09:45.131086620Z 09:09:45,175 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: snapshot is durable as of 2026-03-18T09:09:45.131855393Z 09:09:45,176 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddReplicasForAllShards: snapshot is durable as of 2026-03-18T09:09:45.159428264Z 09:09:45,177 AM [cluster-test-shard-dispatcher-30] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot 09:09:45,177 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:09:45,180 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-2-shard-people-config_testAddReplicasForAllShards: snapshot is durable as of 2026-03-18T09:09:45.159333220Z 09:09:45,180 AM [cluster-test-shard-dispatcher-25] [INFO] ShardSnapshotCohort - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot 09:09:45,180 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-2-shard-people-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:09:45,190 AM [cluster-test-shard-dispatcher-10] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: snapshot is durable as of 2026-03-18T09:09:45.135802613Z 09:09:45,191 AM [cluster-test-shard-dispatcher-13] [INFO] SnapshotManager - member-2-shard-people-oper_testAddReplicasForAllShards: snapshot is durable as of 2026-03-18T09:09:45.162150792Z 09:09:45,191 AM [cluster-test-shard-dispatcher-13] [INFO] ShardSnapshotCohort - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot 09:09:45,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-2-shard-people-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:09:45,191 AM [cluster-test-shard-dispatcher-24] [INFO] SnapshotManager - member-2-shard-cars-config_testAddReplicasForAllShards: snapshot is durable as of 2026-03-18T09:09:45.158723424Z 09:09:45,191 AM [cluster-test-shard-dispatcher-24] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot 09:09:45,192 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:09:45,199 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-2-shard-pets-config_testAddReplicasForAllShards: snapshot is durable as of 2026-03-18T09:09:45.160829315Z 09:09:45,199 AM [cluster-test-shard-dispatcher-31] [INFO] ShardSnapshotCohort - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot 09:09:45,199 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-pets-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:09:45,211 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#2109370609] with name member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher 09:09:45,213 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddReplicasForAllShards: state snapshot applied in 35.60 ms 09:09:45,213 AM [cluster-test-shard-dispatcher-30] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddReplicasForAllShards: Done applying snapshot 09:09:45,216 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#242201972] 09:09:45,216 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#2109370609] 09:09:45,218 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 09:09:45,218 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#-945538027] 09:09:45,218 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-pets-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#-945538027] with name member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher 09:09:45,219 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-pets-config_testAddReplicasForAllShards: publishChanges: rootPath: / 09:09:45,219 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-pets-config_testAddReplicasForAllShards: state snapshot applied in 19.46 ms 09:09:45,219 AM [cluster-test-shard-dispatcher-31] [INFO] ShardSnapshotCohort - member-2-shard-pets-config_testAddReplicasForAllShards: Done applying snapshot 09:09:45,221 AM [cluster-test-shard-dispatcher-10] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:09:45,221 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#855095252] with name member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher 09:09:45,222 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#855095252] 09:09:45,222 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddReplicasForAllShards: state snapshot applied in 30.12 ms 09:09:45,222 AM [cluster-test-shard-dispatcher-24] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddReplicasForAllShards: Done applying snapshot 09:09:45,222 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddReplicasForAllShards: publishChanges: rootPath: / 09:09:45,222 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#242201972] with name member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher 09:09:45,222 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-config_testAddReplicasForAllShards: publishChanges: rootPath: / 09:09:45,222 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-people-config_testAddReplicasForAllShards: state snapshot applied in 42.22 ms 09:09:45,222 AM [cluster-test-shard-dispatcher-25] [INFO] ShardSnapshotCohort - member-2-shard-people-config_testAddReplicasForAllShards: Done applying snapshot 09:09:45,223 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#-1524617818] with name member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher 09:09:45,223 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-2-shard-people-oper_testAddReplicasForAllShards: state snapshot applied in 31.88 ms 09:09:45,223 AM [cluster-test-shard-dispatcher-13] [INFO] ShardSnapshotCohort - member-2-shard-people-oper_testAddReplicasForAllShards: Done applying snapshot 09:09:45,223 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#-1524617818] 09:09:45,223 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 09:09:45,237 AM [cluster-test-shard-dispatcher-13] [INFO] AbstractLeader - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:09:45,237 AM [cluster-test-shard-dispatcher-26] [INFO] AbstractLeader - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:09:45,243 AM [cluster-test-shard-dispatcher-10] [INFO] AbstractLeader - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:09:45,244 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher: 20.88 ms 09:09:45,244 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher: 26.51 ms 09:09:45,244 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher: 22.17 ms 09:09:45,244 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher: 22.08 ms 09:09:45,245 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-pets-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher: 26.00 ms 09:09:45,252 AM [cluster-test-shard-dispatcher-25] [INFO] AbstractLeader - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-pets-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:09:45,258 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#1549188700 09:09:45,258 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 09:09:45,258 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-config_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#909723195 09:09:45,258 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard people 09:09:45,259 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddReplicasForAllShards 09:09:45,259 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 09:09:45,262 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-config_testAddReplicasForAllShards 09:09:45,262 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 09:09:45,263 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 09:09:45,263 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:45,263 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4e3b8732 09:09:45,263 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5b9b8de8 09:09:45,263 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-oper_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#784407353 09:09:45,263 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard people 09:09:45,263 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 09:09:45,263 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:45,263 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 09:09:45,263 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 2 09:09:45,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-oper_testAddReplicasForAllShards 09:09:45,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars, people] 09:09:45,265 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 09:09:45,266 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:45,266 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-1462039311 09:09:45,266 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 09:09:45,266 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done true 09:09:45,266 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddReplicasForAllShards 09:09:45,266 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done true 09:09:45,266 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars, people] 09:09:45,267 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 09:09:45,267 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:45,267 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 09:09:45,267 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 3 09:09:45,267 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 09:09:45,267 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 4 09:09:45,268 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@465ad582 09:09:45,268 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:09:45,268 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7ed108ed 09:09:45,268 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:09:45,268 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2b39a150 09:09:45,268 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 09:09:45,268 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done true 09:09:45,268 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done true 09:09:45,268 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done true 09:09:45,270 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 09:09:45,270 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 5 09:09:45,270 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 09:09:45,270 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 09:09:45,271 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 09:09:45,271 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 09:09:45,274 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-pets-config_testAddReplicasForAllShards} for shard pets from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#-528388922 09:09:45,274 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard pets 09:09:45,270 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorVotingConfigSupport - member-1-shard-pets-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 09:09:45,278 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 09:09:45,280 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 09:09:45,280 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-pets-config_testAddReplicasForAllShards 09:09:45,281 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 09:09:45,281 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:45,282 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars, people] 09:09:45,285 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 09:09:45,285 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=pets}, type Config 09:09:45,285 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 09:09:45,285 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 6 09:09:45,292 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2080736686] 09:09:45,292 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:09:45,295 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#-1558583254] 09:09:45,295 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#-1558583254] 09:09:45,296 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2080736686] 09:09:45,296 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:09:45,297 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#1562414916] 09:09:45,297 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#1562414916] 09:09:45,297 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#-2080736686] 09:09:45,297 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 09:09:45,298 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-633032912] 09:09:45,298 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#-633032912] 09:09:45,298 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#382209312] 09:09:45,298 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:09:45,298 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-208615085] 09:09:45,299 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-208615085] 09:09:45,299 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#382209312] 09:09:45,299 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 09:09:45,299 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#857334234] 09:09:45,299 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#857334234] 09:09:45,300 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 09:09:45,301 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testAddReplicasForAllShards 09:09:45,301 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddReplicasForAllShards 09:09:45,301 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testAddReplicasForAllShards 09:09:45,301 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 09:09:45,302 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Stopping Shard 09:09:45,302 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Stopping Shard 09:09:45,303 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Stopping Shard 09:09:45,303 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-config_testAddReplicasForAllShards: journal closed 09:09:45,303 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddReplicasForAllShards: journal closed 09:09:45,304 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testAddReplicasForAllShards: journal closed 09:09:45,305 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:09:45,305 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:09:45,306 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 09:09:45,306 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddReplicasForAllShards 09:09:45,307 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testAddReplicasForAllShards 09:09:45,307 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 09:09:45,307 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 3 shards 09:09:45,308 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Stopping Shard 09:09:45,308 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Stopping Shard 09:09:45,308 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Stopping Shard 09:09:45,308 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-2-shard-no-leader-oper_testAddReplicasForAllShards: journal closed 09:09:45,308 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testAddReplicasForAllShards: journal closed 09:09:45,309 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddReplicasForAllShards: journal closed 09:09:45,309 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:09:45,309 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:09:45,311 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:09:45,311 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:09:45,346 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:09:45,348 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:09:45,355 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:09:45,373 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 09:09:45,374 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testAddReplicasForAllShards 09:09:45,375 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddReplicasForAllShards 09:09:45,375 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testAddReplicasForAllShards 09:09:45,375 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 09:09:45,378 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4546ae36 09:09:45,378 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5a5acdcc 09:09:45,378 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3d0d9fc4 09:09:45,379 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1e558f80 09:09:45,379 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2f2eafde 09:09:45,379 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1e558f80 09:09:45,379 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2f2eafde 09:09:45,379 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 09:09:45,379 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7adb4ddd 09:09:45,379 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 09:09:45,379 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-pets-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7adb4ddd 09:09:45,380 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 09:09:46,345 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [5223643864626453675] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:09:49,577 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:49,895 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:50,46 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:50,56 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:50,534 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:50,859 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:51,64 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 09:09:51,68 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:09:51,68 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:09:51,75 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:51,75 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:51,497 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:51,817 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:52,17 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:52,96 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:52,456 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:53,93 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:53,114 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:54,25 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:54,136 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:09:54,363 AM [cluster-test-shard-dispatcher-13] [WARN] Leader - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 09:09:54,363 AM [cluster-test-shard-dispatcher-10] [WARN] Leader - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 09:09:54,364 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:09:54,364 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:09:54,364 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer aborted 09:09:54,364 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:09:54,364 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: Failed to transfer leadership in 8.987 s 09:09:54,365 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In unpauseLeader 09:09:54,365 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 09:09:54,365 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:09:54,365 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 09:09:54,365 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 09:09:54,365 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 09:09:54,366 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Stopping Shard 09:09:54,366 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddReplicasForAllShards: journal closed 09:09:54,373 AM [cluster-test-shard-dispatcher-13] [WARN] Leader - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-pets-config_testAddReplicasForAllShards from Leader to IsolatedLeader 09:09:54,373 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:09:54,373 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer aborted 09:09:54,373 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: Failed to transfer leadership in 8.995 s 09:09:54,373 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In unpauseLeader 09:09:54,373 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:09:54,373 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 09:09:54,374 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Leader to IsolatedLeader 09:09:54,374 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Stopping Shard 09:09:54,375 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testAddReplicasForAllShards: journal closed 09:09:54,382 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer aborted 09:09:54,382 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.005 s 09:09:54,382 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In unpauseLeader 09:09:54,383 AM [cluster-test-shard-dispatcher-13] [WARN] Leader - member-1-shard-people-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 09:09:54,383 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:09:54,384 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:09:54,384 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 09:09:54,384 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 09:09:54,384 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Stopping Shard 09:09:54,384 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-config_testAddReplicasForAllShards: journal closed 09:09:54,385 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:09:54,385 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:09:54,386 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 09:09:54,387 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddReplicasForAllShards 09:09:54,387 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testAddReplicasForAllShards 09:09:54,387 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:09:54,387 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Stopping Shard 09:09:54,387 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6a16f846 09:09:54,387 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@9a82fc6 09:09:54,388 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-1-shard-people-oper_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@9a82fc6 09:09:54,388 AM [cluster-test-shard-dispatcher-10] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 09:09:54,388 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddReplicasForAllShards: journal closed 09:09:54,393 AM [cluster-test-shard-dispatcher-13] [WARN] Leader - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 09:09:54,393 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:09:54,393 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testAddReplicasForAllShards: leader transfer aborted 09:09:54,393 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testAddReplicasForAllShards: Failed to transfer leadership in 6.016 ms 09:09:54,393 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: In unpauseLeader 09:09:54,393 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:09:54,393 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 09:09:54,394 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 09:09:54,394 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Stopping Shard 09:09:54,395 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testAddReplicasForAllShards: journal closed 09:09:54,395 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:09:54,395 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:09:54,397 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:09:54,397 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:09:54,400 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:09:54.404] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:09:54,579 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:09:54,611 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [3494082824927048293] 09:09:54,612 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:09:54,615 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:09:54,616 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:09:54,619 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:09:54,619 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:09:54,640 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:54,641 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:09:54,641 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:09:54,642 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:09:54,645 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:54,651 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 09:09:54,655 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1622172073] 09:09:54,656 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1622172073] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,656 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 09:09:54,657 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:54,660 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:09:54,660 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:54,660 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:09:54,660 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:09:54,660 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStates 09:09:54,661 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStates 09:09:54,661 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@5bd6e6d6} 09:09:54,668 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:54,668 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStates 09:09:54,669 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 09:09:54,671 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStates 09:09:54,672 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:54,672 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:09:54,672 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:09:54,672 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:09:54,672 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:09:54,673 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:54,673 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1744824185] 09:09:54,673 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 09:09:54,673 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1744824185] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,674 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,674 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 09:09:54,675 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 09:09:54,675 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 09:09:54,677 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#-1186535932 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStates 09:09:54,677 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#-1186535932] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,677 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#217745718 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStates 09:09:54,678 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#217745718] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,678 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 09:09:54,680 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1358173679] 09:09:54,681 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1358173679] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,685 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:09:54,685 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:09:54,685 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:09:54,685 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:54,685 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStates 09:09:54,686 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@73c9f82f} 09:09:54,686 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:54,686 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:54,687 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 09:09:54,687 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 09:09:54,693 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStates: Recovery completed in in 5.318 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:09:54,695 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStates: Recovery completed in in 7.514 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:09:54,695 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:54,695 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:54,696 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:09:54,696 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStates is initialized 09:09:54,696 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:09:54,696 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStates is initialized 09:09:54,697 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 09:09:54,697 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStates 09:09:54,698 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:54,698 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:54,698 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:54,698 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:09:54,698 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:09:54,698 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#878366306] 09:09:54,698 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 09:09:54,699 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#878366306] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,699 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 09:09:54,699 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#1489481249 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:54,700 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#1489481249] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,701 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 09:09:54,702 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Recovery completed in in 1.007 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:09:54,702 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:54,702 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:54,702 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 09:09:54,702 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-469831809] 09:09:54,703 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-469831809] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,703 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 09:09:54,703 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 09:09:54,703 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:09:54,703 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStates is initialized 09:09:54,703 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#2031386353 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStates 09:09:54,704 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 09:09:54,704 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#2031386353] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,709 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 09:09:54,710 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 09:09:54,710 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:54,710 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:54,710 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 09:09:54,710 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 09:09:54,710 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from null to Follower 09:09:54,710 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from null to Follower 09:09:54,711 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStates: Recovery completed in in 2.263 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:09:54,712 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:54,712 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:54,713 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 09:09:54,714 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from null to Follower 09:09:54,715 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:09:54,715 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStates is initialized 09:09:54,715 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 09:09:54,715 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from null to Follower 09:09:54,716 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:09:54,734 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-4352430779084673176] 09:09:54,736 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.4.0] ... 09:09:54,740 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:09:54,743 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:09:54,743 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:09:54,743 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:09:54,765 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:54,769 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:54,773 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 09:09:54,788 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 09:09:54,789 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:54,790 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1601958048] 09:09:54,792 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:54,792 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1601958048] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,793 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:09:54,794 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 09:09:54,794 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 09:09:54,794 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStates 09:09:54,796 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStates 09:09:54,796 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@7bcf0f6} 09:09:54,797 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 09:09:54,799 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-2059727535] 09:09:54,799 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-2059727535] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,800 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:09:54,800 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 09:09:54,800 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 09:09:54,800 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStates 09:09:54,800 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStates 09:09:54,801 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@5e667155} 09:09:54,802 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 09:09:54,807 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:54,807 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStates 09:09:54,808 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:54,808 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStates 09:09:54,818 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStates 09:09:54,819 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStates 09:09:54,819 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-1340370237] 09:09:54,819 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-1340370237] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,819 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 09:09:54,819 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1565976267] 09:09:54,820 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1565976267] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,820 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 09:09:54,821 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#290923442 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStates 09:09:54,821 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#290923442] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,821 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 09:09:54,821 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#2024023009 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStates 09:09:54,821 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 09:09:54,823 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:09:54,823 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 09:09:54,823 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 09:09:54,824 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Recovery completed in in 1.243 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:09:54,825 AM [cluster-test-shard-dispatcher-25] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStates: Recovery completed in in 1.251 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:09:54,825 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:54,825 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:54,825 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 09:09:54,825 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-170074547] 09:09:54,825 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:54,825 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:54,825 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 09:09:54,825 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1174350907] 09:09:54,825 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 09:09:54,826 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 09:09:54,826 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#-1079087472 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStates 09:09:54,826 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 09:09:54,827 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#-1909910400 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStates 09:09:54,827 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 09:09:54,828 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 09:09:54,828 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 09:09:54,829 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStates: Recovery completed in in 881.7 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:09:54,829 AM [cluster-test-shard-dispatcher-25] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStates: Recovery completed in in 933.5 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:09:54,830 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:09:54,830 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStates is initialized 09:09:54,830 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:54,830 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:54,830 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 09:09:54,830 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:54,830 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:54,830 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 09:09:54,831 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 09:09:54,833 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:09:54,836 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:09:54,839 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:09:54,839 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStates is initialized 09:09:54,840 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from null to Follower 09:09:54,840 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:54,840 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 09:09:54,840 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:54,840 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:09:54,840 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:09:54,840 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:09:54,840 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:54,840 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from null to Follower 09:09:54,840 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:09:54,840 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#2024023009] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,841 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-170074547] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,841 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1174350907] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,841 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#-1079087472] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,841 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#-1909910400] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,841 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:09:54,841 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStates is initialized 09:09:54,841 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:09:54,841 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStates is initialized 09:09:54,841 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:54,841 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:09:54,841 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:09:54,841 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:09:54,841 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:09:54,841 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:09:54,841 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:09:54,842 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 09:09:54,842 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 09:09:54,842 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from null to Follower 09:09:54,842 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from null to Follower 09:09:54,855 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-5328544819434497575] 09:09:54,856 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.4.0] ... 09:09:54,861 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:09:54,861 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:09:54,862 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:09:54,863 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:09:54,877 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:54,879 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:54,882 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 09:09:54,906 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#2083585467] 09:09:54,908 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:09:54,908 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 09:09:54,908 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 09:09:54,908 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStates 09:09:54,909 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStates 09:09:54,909 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#2083585467] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,909 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 09:09:54,910 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@1cf326bf} 09:09:54,910 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:54,910 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:54,910 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStates 09:09:54,912 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:54,915 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 09:09:54,917 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStates 09:09:54,918 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#479705441] 09:09:54,919 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 09:09:54,920 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#479705441] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,921 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#930155451] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,921 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#930155451 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStates 09:09:54,927 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 09:09:54,929 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#226950134] 09:09:54,929 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 09:09:54,930 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#-942824018 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStates 09:09:54,931 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#226950134] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,931 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 09:09:54,931 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#-942824018] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,932 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:09:54,934 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 09:09:54,936 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStates: Recovery completed in in 885.6 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:09:54,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:54,937 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:54,937 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 09:09:54,937 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:09:54,939 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1955332529] 09:09:54,940 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1955332529] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,941 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:09:54,941 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 09:09:54,941 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 09:09:54,941 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:54,941 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:09:54,945 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStates 09:09:54,945 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@4741e196} 09:09:54,945 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:54,945 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:54,947 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 09:09:54,947 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:09:54,948 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStates is initialized 09:09:54,948 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-746391201] 09:09:54,948 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-746391201] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,948 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 09:09:54,948 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 09:09:54,949 AM [cluster-test-shard-dispatcher-22] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStates: Recovery completed in in 849.8 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:09:54,950 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#-774844994] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,950 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:54,950 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:54,950 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 09:09:54,950 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 09:09:54,950 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#-774844994 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:54,950 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 09:09:54,954 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 09:09:54,954 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:54,954 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:09:54,954 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:09:54,954 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:09:54,955 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:09:54,955 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:09:54,955 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:09:54,955 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Recovery completed in in 823.8 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:09:54,956 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:54,956 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:54,956 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 09:09:54,956 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:09:54,956 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStates is initialized 09:09:54,956 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from null to Follower 09:09:54,956 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 09:09:54,956 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from null to Follower 09:09:54,957 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStates 09:09:54,958 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:54,958 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:54,958 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:54,958 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:09:54,958 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:09:54,958 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-938453393] 09:09:54,958 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 09:09:54,958 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-938453393] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,959 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:54,959 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:09:54,959 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStates is initialized 09:09:54,959 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 09:09:54,959 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from null to Follower 09:09:54,959 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#258047806 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStates 09:09:54,959 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 09:09:54,959 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#258047806] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:54,961 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 09:09:54,962 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStates: Recovery completed in in 908.3 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:09:54,965 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:09:54,965 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStates is initialized 09:09:54,965 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:54,965 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:54,965 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 09:09:54,966 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:09:54,966 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 09:09:54,966 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from null to Follower 09:09:55,632 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:09:55,632 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:09:55,633 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:09:55,633 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:09:55,633 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:09:55,633 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:09:55,634 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:09:55,634 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:09:55,634 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 09:09:55,635 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:09:55,635 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:55,635 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:09:55,635 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:09:55,635 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:09:55,635 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:09:55,634 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:09:55,635 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 09:09:55,635 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:09:55,653 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:09:55,653 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:09:55,653 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:09:55,653 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:09:55,653 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:55,654 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:09:55,654 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:09:55,655 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:09:55,655 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:09:55,764 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 09:09:55,764 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:09:55,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:55,765 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:55,765 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:55,765 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 09:09:55,776 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 09:09:55,777 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:09:55,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:55,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:55,777 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:55,777 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 09:09:55,777 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:09:55,780 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:09:55,784 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 09:09:55,785 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:09:55,785 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:55,785 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:55,785 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:55,785 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 09:09:55,792 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 09:09:55,793 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:09:55,793 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:55,793 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:55,793 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:55,794 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 09:09:55,794 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:09:55,794 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:55,794 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:55,794 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:09:55,794 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:09:55,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:09:55,794 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:09:55,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:09:55,794 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:09:55,794 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:09:55,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:09:55,794 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:09:55,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:09:55,795 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 09:09:55,795 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 09:09:55,796 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:09:55,796 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:09:55,796 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:09:55,796 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:09:55,796 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:09:55,796 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:55,796 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:09:55,796 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:09:55,796 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 09:09:55,796 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 09:09:55,799 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:09:55,800 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:09:55,801 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:09:55,803 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:09:55,803 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:55,804 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:55,804 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 09:09:55,804 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c177188 09:09:55,804 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:09:55,804 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:55,805 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:55,806 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 09:09:55,806 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5abf811e 09:09:55,806 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to PreLeader 09:09:55,804 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to PreLeader 09:09:55,809 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 09:09:55,810 AM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 09:09:55,812 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1234cb51 09:09:55,812 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done false 09:09:55,812 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done true 09:09:55,813 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done false 09:09:55,813 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done true 09:09:55,813 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@160b4dcf 09:09:55,814 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5d2b83b 09:09:55,814 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 09:09:55,817 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:09:55,817 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:09:55,817 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:09:55,817 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:09:55,817 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:55,817 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:55,817 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:55,818 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:55,818 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 09:09:55,818 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 09:09:55,818 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@54fb6c4f 09:09:55,818 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to PreLeader 09:09:55,818 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@18d96fbf 09:09:55,821 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to PreLeader 09:09:55,822 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@353f8277 09:09:55,823 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 09:09:55,825 AM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 09:09:55,826 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 09:09:55,827 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 09:09:55,827 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 09:09:55,832 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 09:09:55,832 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 09:09:55,832 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4418d1f2 09:09:55,832 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6b212b76 09:09:55,832 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:09:55,832 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:09:55,832 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done false 09:09:55,832 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done true 09:09:55,832 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3bf86aa0 09:09:55,833 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:09:55,833 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 09:09:55,833 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 09:09:55,833 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@97b1e71 09:09:55,833 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:09:55,833 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 09:09:55,833 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 09:09:55,844 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:09:55,844 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:55,844 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 09:09:55,844 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from PreLeader to Leader 09:09:55,845 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:09:55,846 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:55,846 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 09:09:55,846 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from PreLeader to Leader 09:09:55,853 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:09:55,853 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:55,853 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 09:09:55,853 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from PreLeader to Leader 09:09:55,853 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:09:55,854 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:09:55,854 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:55,854 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 09:09:55,854 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from PreLeader to Leader 09:09:55,854 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:09:55,855 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 09:09:55,855 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:09:55,855 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 09:09:55,855 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:09:55,855 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 09:09:55,855 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:09:55,855 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 09:09:55,856 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1622172073] 09:09:55,856 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:55,856 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:55,856 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:55,861 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 09:09:55,861 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 09:09:55,861 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 09:09:55,861 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:09:55,861 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:09:55,863 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:09:55,863 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:09:55,863 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:09:55,863 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:09:55,863 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:09:55,863 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:09:55,866 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:09:55,867 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:09:55,870 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:09:55,870 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:55,879 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:09:55,898 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:09:55,898 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 09:09:55,898 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 09:09:55,898 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 09:09:55,899 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:09:55,899 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 09:09:55,899 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 09:09:55,899 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 09:09:55,899 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:09:55,899 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 09:09:55,899 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 09:09:55,899 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 09:09:55,902 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:55,902 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 09:09:55,902 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 09:09:55,902 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 09:09:55,904 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1622172073] 09:09:55,904 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:55,904 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:55,904 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:55,905 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1622172073] 09:09:55,908 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:09:55,908 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1744824185] 09:09:55,908 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1744824185] 09:09:55,910 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1358173679] 09:09:55,911 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:09:55,911 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#878366306] 09:09:55,911 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#878366306] 09:09:55,911 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1358173679] 09:09:55,911 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:09:55,911 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-469831809] 09:09:55,912 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-469831809] 09:09:55,912 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1601958048] 09:09:55,912 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:09:55,912 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-1340370237] 09:09:55,913 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 09:09:55,914 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 09:09:55,914 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7b95e788 09:09:55,914 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7b95e788 09:09:55,914 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:55,915 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 09:09:55,915 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7552068a 09:09:55,915 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7552068a 09:09:55,915 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:55,916 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 09:09:55,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@716d8cf8 09:09:55,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@716d8cf8 09:09:55,916 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:55,916 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@346f0cb1 09:09:55,917 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@60920413 09:09:55,917 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@51138dd 09:09:55,917 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-1340370237] 09:09:55,918 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1601958048] 09:09:55,918 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:09:55,918 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1174350907] 09:09:55,918 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@51faf81 09:09:55,919 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1174350907] 09:09:55,919 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-2059727535] 09:09:55,919 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:09:55,919 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1565976267] 09:09:55,919 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@71e37138 09:09:55,919 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@71e37138 09:09:55,920 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:55,920 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1565976267] 09:09:55,922 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-2059727535] 09:09:55,922 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 09:09:55,922 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-170074547] 09:09:55,923 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-170074547] 09:09:55,923 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#2083585467] 09:09:55,923 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:09:55,923 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#479705441] 09:09:55,923 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#479705441] 09:09:55,925 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#2083585467] 09:09:55,925 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:09:55,925 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#226950134] 09:09:55,925 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#226950134] 09:09:55,926 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1955332529] 09:09:55,926 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:09:55,927 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-746391201] 09:09:55,927 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-746391201] 09:09:55,927 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1955332529] 09:09:55,927 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:09:55,927 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-938453393] 09:09:55,927 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-938453393] 09:09:55,928 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1622172073] 09:09:55,928 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:55,928 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:55,929 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:55,934 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7780532a 09:09:55,935 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@29db3971 09:09:55,935 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4cd24173 09:09:55,935 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:09:55,939 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:09:55,943 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@743c3d5e 09:09:55,946 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:09:55,947 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1193f22c 09:09:55,947 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@56a7a59b 09:09:55,956 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@353d3626 09:09:55,958 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3768e1b2 09:09:55,963 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 09:09:55,963 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 09:09:55,963 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:55,963 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:55,964 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:55,964 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:09:55,964 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@49b09b50 09:09:55,964 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 09:09:55,964 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:09:55,964 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:55,964 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@55de9d0c 09:09:55,964 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:55,965 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 09:09:55,965 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 09:09:55,969 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 09:09:55,969 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 09:09:55,969 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:55,969 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:55,970 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:55,970 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 09:09:55,970 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 09:09:55,970 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:55,970 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:55,970 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@39d2522f 09:09:55,970 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 09:09:55,970 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:09:55,970 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:55,970 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:55,970 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:55,970 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 09:09:55,970 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6144bd25 09:09:55,971 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 09:09:55,971 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@11a410cd 09:09:55,971 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 09:09:55,971 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:09:55,971 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:09:55,971 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:55,971 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:55,971 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c8373b2 09:09:55,971 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 09:09:55,971 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 09:09:55,976 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 09:09:55,977 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 09:09:55,977 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 09:09:55,976 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 09:09:55,978 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 09:09:55,978 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:55,978 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:55,978 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:55,978 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7d94a3d3 09:09:55,978 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 09:09:55,979 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:09:55,979 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:55,979 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:55,979 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@397d84d1 09:09:55,979 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 09:09:55,979 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@10dfe6f4 09:09:55,980 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 09:09:55,980 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 09:09:55,980 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:09:55,981 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 09:09:55,981 AM [cluster-test-shard-dispatcher-13] [INFO] AbstractLeader - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 09:09:55,981 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@226830fa 09:09:55,982 AM [cluster-test-shard-dispatcher-16] [INFO] AbstractLeader - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 09:09:55,983 AM [cluster-test-shard-dispatcher-15] [INFO] AbstractLeader - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 09:09:55,984 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 09:09:55,985 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2cafcac7 09:09:55,985 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:09:55,985 AM [cluster-test-shard-dispatcher-24] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 09:09:55,987 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 09:09:55,987 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:55,987 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:55,987 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 09:09:55,987 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:55,987 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:55,987 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 09:09:55,987 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 09:09:55,987 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@bd85152 09:09:55,987 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5d8bad7a 09:09:55,987 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 09:09:55,987 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Leader to Follower 09:09:55,987 AM [cluster-test-shard-dispatcher-13] [INFO] AbstractLeader - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 09:09:55,988 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:55,988 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 09:09:55,988 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:55,988 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:55,989 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 09:09:55,989 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@77ed894d 09:09:55,989 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 09:09:55,990 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@647fea37 09:09:55,990 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:09:55,990 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 09:09:55,990 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 09:09:55,990 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:55,990 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:55,990 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 09:09:55,991 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@8789be7 09:09:55,991 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 09:09:55,995 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 09:09:55,995 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 09:09:55,995 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 09:09:55,996 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 09:09:56,39 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:56,90 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:56,141 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:56,193 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:56,244 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:56,284 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader changed to member-3-shard-cars-config_testFlipMemberVotingStates 09:09:56,284 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-config_testFlipMemberVotingStates in 365.9 ms 09:09:56,284 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3f5384ff 09:09:56,285 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 09:09:56,292 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 09:09:56,293 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader changed to member-3-shard-people-config_testFlipMemberVotingStates 09:09:56,294 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7c7d664b 09:09:56,294 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@60c99e5 09:09:56,294 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:09:56,294 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:56,295 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-oper_testFlipMemberVotingStates in 380.5 ms 09:09:56,295 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:56,295 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 09:09:56,295 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-config_testFlipMemberVotingStates in 380.3 ms 09:09:56,296 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 09:09:56,296 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1358173679] 09:09:56,296 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:09:56,296 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#878366306] 09:09:56,296 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#878366306] 09:09:56,296 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 09:09:56,297 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 09:09:56,297 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 09:09:56,297 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:09:56,297 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:09:56,298 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:09:56,298 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:09:56,298 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:09:56,298 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:09:56,298 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:09:56,298 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:09:56,301 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:56,301 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:09:56,301 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:09:56,303 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 09:09:56,303 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 09:09:56,303 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 09:09:56,303 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 09:09:56,303 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 09:09:56,303 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 09:09:56,303 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 09:09:56,303 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 09:09:56,304 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:56,304 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 09:09:56,304 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 09:09:56,304 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:09:56,304 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 09:09:56,305 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 09:09:56,305 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 09:09:56,305 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 09:09:56,305 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 09:09:56,305 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1622172073] 09:09:56,305 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:56,307 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:56,307 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:56,308 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader changed to member-3-shard-people-oper_testFlipMemberVotingStates 09:09:56,308 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-oper_testFlipMemberVotingStates in 392.5 ms 09:09:56,309 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@18f5b18d 09:09:56,309 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:09:56,309 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 09:09:56,309 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 09:09:56,310 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 09:09:56,310 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 09:09:56,311 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorVotingConfigSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 09:09:56,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@731f04b7 09:09:56,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@731f04b7 09:09:56,311 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:56,312 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorVotingConfigSupport - member-3-shard-people-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 09:09:56,312 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@77208e0c 09:09:56,312 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-3-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@77208e0c 09:09:56,312 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:56,315 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@32220e8c 09:09:56,315 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@8ebfeb1 09:09:56,316 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-3-shard-cars-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 09:09:56,316 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@8eec94b 09:09:56,316 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@8eec94b 09:09:56,316 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:56,317 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@37aa41d2 09:09:56,317 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@622aae42 09:09:56,320 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5bb32482 09:09:56,320 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@58389cb4 09:09:56,321 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-3-shard-people-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 09:09:56,321 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:09:56,321 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4f373507 09:09:56,321 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4f373507 09:09:56,321 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:56,321 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:09:56,322 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:09:56,322 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@35a79a7 09:09:56,323 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5ac6596c 09:09:56,323 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7079d72e 09:09:56,331 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2dfbdb67 09:09:56,332 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:09:56,332 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2ec71937 09:09:56,335 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@773fbe47 09:09:56,335 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 09:09:56,335 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:09:56,336 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,336 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,336 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:56,336 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@c73e968 09:09:56,336 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 09:09:56,337 AM [cluster-test-shard-dispatcher-13] [INFO] Follower - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 09:09:56,343 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 09:09:56,343 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:09:56,343 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,343 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,343 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:56,344 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@25c6f41b 09:09:56,344 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 09:09:56,345 AM [cluster-test-shard-dispatcher-18] [INFO] Follower - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 09:09:56,347 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 09:09:56,347 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-2-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:09:56,347 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,347 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,347 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:56,347 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 09:09:56,348 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:09:56,348 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,348 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,347 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@dbd3fb3 09:09:56,348 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 09:09:56,348 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:56,348 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5438ca07 09:09:56,349 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 09:09:56,349 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-1-shard-cars-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 09:09:56,350 AM [cluster-test-shard-dispatcher-24] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 09:09:56,358 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:56,364 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:09:56,364 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:56,364 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:56,364 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 09:09:56,364 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@21b270b9 09:09:56,364 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 09:09:56,369 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:09:56,369 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:56,369 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:56,369 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 09:09:56,369 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@37a55ce6 09:09:56,369 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 09:09:56,370 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 09:09:56,370 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@75820811 09:09:56,371 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1622172073] 09:09:56,372 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:09:56,372 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1744824185] 09:09:56,372 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 09:09:56,373 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@165c0624 09:09:56,373 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1744824185] 09:09:56,374 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-2-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:09:56,374 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:56,374 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:56,374 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 09:09:56,374 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7549c022 09:09:56,374 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 09:09:56,374 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:09:56,374 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1358173679] 09:09:56,375 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:09:56,375 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#878366306] 09:09:56,375 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#878366306] 09:09:56,375 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1358173679] 09:09:56,375 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:09:56,375 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-469831809] 09:09:56,375 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#-469831809] 09:09:56,375 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1601958048] 09:09:56,376 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:09:56,376 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-1340370237] 09:09:56,376 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#-1340370237] 09:09:56,376 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#1601958048] 09:09:56,377 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:09:56,378 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1174350907] 09:09:56,378 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#-1174350907] 09:09:56,378 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:09:56,378 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:56,378 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:56,378 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 09:09:56,378 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 09:09:56,378 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 09:09:56,378 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 09:09:56,378 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@fe7c4f1 09:09:56,378 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@40ff03b9 09:09:56,379 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 09:09:56,379 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:09:56,379 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:09:56,379 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-2059727535] 09:09:56,380 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:09:56,380 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1565976267] 09:09:56,380 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1565976267] 09:09:56,380 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-2059727535] 09:09:56,380 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 09:09:56,380 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-170074547] 09:09:56,381 AM [cluster-test-shard-dispatcher-26] [INFO] AbstractLeader - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 09:09:56,381 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#-170074547] 09:09:56,382 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#2083585467] 09:09:56,382 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:09:56,382 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#479705441] 09:09:56,382 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#479705441] 09:09:56,383 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#2083585467] 09:09:56,383 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 09:09:56,383 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:09:56,383 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#226950134] 09:09:56,383 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@240ecbd3 09:09:56,383 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:09:56,383 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#226950134] 09:09:56,384 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1955332529] 09:09:56,384 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:09:56,384 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-746391201] 09:09:56,384 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#-746391201] 09:09:56,385 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#1955332529] 09:09:56,385 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:09:56,385 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 09:09:56,385 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-938453393] 09:09:56,385 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#-938453393] 09:09:56,385 AM [cluster-test-shard-dispatcher-27] [INFO] AbstractLeader - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 09:09:56,386 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 09:09:56,387 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 09:09:56,387 AM [cluster-test-shard-dispatcher-24] [INFO] AbstractLeader - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 09:09:56,394 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 09:09:56,394 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,394 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,395 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 09:09:56,395 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@62b6b1cb 09:09:56,395 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 09:09:56,395 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1622172073] 09:09:56,395 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:56,395 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:56,395 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1456359886] 09:09:56,396 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:09:56,397 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStates 09:09:56,397 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStates 09:09:56,397 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 09:09:56,397 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,397 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 09:09:56,397 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,398 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 09:09:56,398 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,398 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,398 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 09:09:56,398 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 09:09:56,399 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Stopping Shard 09:09:56,399 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStates: journal closed 09:09:56,399 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 09:09:56,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,399 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 09:09:56,399 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2173659f 09:09:56,399 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@520df1c3 09:09:56,400 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Leader to Follower 09:09:56,400 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 09:09:56,400 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStates: journal closed 09:09:56,400 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:09:56,401 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:09:56,401 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:09:56,402 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStates 09:09:56,403 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStates 09:09:56,403 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:09:56,403 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 09:09:56,403 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 09:09:56,403 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStates: journal closed 09:09:56,404 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStates: journal closed 09:09:56,404 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:09:56,404 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:09:56,406 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:09:56,406 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:09:56,415 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:09:56,417 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:09:56,417 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:09:56,417 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:09:56,421 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:09:56,440 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:09:56,443 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStates 09:09:56,444 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStates 09:09:56,444 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:09:56,444 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@367a9565 09:09:56,445 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3040c874 09:09:56,445 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3040c874 09:09:56,445 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:56,445 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ff0cab 09:09:56,445 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7cb66f1 09:09:56,445 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-2-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7cb66f1 09:09:56,445 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:56,449 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@e5eae02 09:09:56,449 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1aa7672 09:09:56,453 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:09:56,453 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:09:56,459 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 5 09:09:56,459 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:09:56,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,459 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,459 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:56,459 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@694f4212 09:09:56,459 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 09:09:56,460 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 5 09:09:56,460 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:09:56,460 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,460 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,460 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:56,460 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@312ba126 09:09:56,460 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 09:09:56,461 AM [cluster-test-shard-dispatcher-24] [INFO] AbstractLeader - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 09:09:56,463 AM [cluster-test-shard-dispatcher-23] [INFO] AbstractLeader - member-2-shard-cars-config_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 09:09:56,469 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 09:09:56,469 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,469 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,469 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 09:09:56,470 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6d57ac01 09:09:56,470 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Leader to Follower 09:09:56,470 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-2-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 09:09:56,470 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,470 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,470 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 09:09:56,470 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@eb99984 09:09:56,470 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 09:09:56,472 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:09:56,472 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:56,472 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:56,472 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 09:09:56,472 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3b6cfcc2 09:09:56,472 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 09:09:56,472 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:09:56,473 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:56,473 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:56,473 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5e4bb93e 09:09:56,473 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 09:09:56,473 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 09:09:56,473 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:09:56,474 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: leader changed to member-1-shard-cars-config_testFlipMemberVotingStates 09:09:56,474 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-config_testFlipMemberVotingStates in 30.16 ms 09:09:56,475 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@19a022be 09:09:56,475 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 09:09:56,475 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: leader changed to member-1-shard-people-config_testFlipMemberVotingStates 09:09:56,475 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-people-config_testFlipMemberVotingStates in 30.43 ms 09:09:56,475 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 09:09:56,475 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4b22220a 09:09:56,475 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 09:09:56,475 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 09:09:56,476 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Stopping Shard 09:09:56,476 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStates: journal closed 09:09:56,476 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStates: journal closed 09:09:56,477 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:09:56,477 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:09:56,479 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:09:56,480 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStates 09:09:56,480 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStates 09:09:56,480 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:09:56,481 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@76d45ab 09:09:56,481 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@76d45ab 09:09:56,481 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:56,481 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@299157b6 09:09:56,481 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 09:09:56,482 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStates: journal closed 09:09:56,484 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7b35c5ce 09:09:56,486 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 09:09:56,492 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 5 09:09:56,492 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:09:56,492 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,492 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,492 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@604a14 09:09:56,493 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 09:09:56,493 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 09:09:56,494 AM [cluster-test-shard-dispatcher-29] [INFO] AbstractLeader - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 09:09:56,498 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 09:09:56,499 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 09:09:56,499 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:56,499 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 09:09:56,499 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@79d3ab98 09:09:56,499 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 09:09:56,501 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:09:56,502 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 09:09:56,502 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@522ba58c 09:09:56,502 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 09:09:56,503 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 09:09:56,503 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 09:09:56,503 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:09:56,504 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:56,504 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-oper_testFlipMemberVotingStates in 24.14 ms 09:09:56,505 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 09:09:56,505 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@55f78c7b 09:09:56,505 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 09:09:56,505 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 09:09:56,506 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStates: journal closed 09:09:56,506 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:09:56,506 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:09:56,508 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:09:56,508 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:09:56,652 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:09:57,420 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-5328544819434497575] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:09:57,528 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:09:57,528 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:09:57,531 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-4352430779084673176] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:09:57,538 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:09:57.540] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:09:57,545 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 09:09:57,546 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStates 09:09:57,546 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStates 09:09:57,546 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:09:57,546 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@28d0e9f5 09:09:57,547 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@441e5f58 09:09:57,547 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@63280325 09:09:57,547 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@63280325 09:09:57,547 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:57,547 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@41e73209 09:09:57,547 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@41e73209 09:09:57,547 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:58,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer aborted 09:09:58,631 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer aborted 09:09:58,631 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.085 s 09:09:58,631 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.085 s 09:09:58,631 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In unpauseLeader 09:09:58,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In unpauseLeader 09:09:58,632 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Stopping Shard 09:09:58,633 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 09:09:58,633 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStates: journal closed 09:09:58,633 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStates: journal closed 09:09:58,634 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:09:58,634 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:09:58,634 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 09:09:58,636 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStates 09:09:58,636 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStates 09:09:58,636 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:09:58,636 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4ae55b69 09:09:58,636 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4ae55b69 09:09:58,636 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:58,636 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1096d14 09:09:58,636 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@18c821f5 09:09:58,636 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@18c821f5 09:09:58,637 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 09:09:58,637 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3445cd61 09:09:59,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer aborted 09:09:59,730 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.094 s 09:09:59,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In unpauseLeader 09:09:59,731 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 09:09:59,732 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStates: journal closed 09:09:59,740 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer aborted 09:09:59,740 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.104 s 09:09:59,741 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In unpauseLeader 09:09:59,741 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 09:09:59,742 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStates: journal closed 09:09:59,743 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:09:59,743 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:09:59,745 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:09:59,745 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:09:59,751 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:09:59.755] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:09:59,787 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:09:59,810 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-2426685009080459311] 09:09:59,811 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:09:59,815 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:09:59,815 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:09:59,824 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:09:59,824 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:09:59,831 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:59,832 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:09:59,832 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:09:59,835 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:59,836 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:09:59,838 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 09:09:59,841 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#916511707] 09:09:59,841 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:09:59,841 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:09:59,842 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:09:59,842 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testModuleShardLeaderMovement 09:09:59,842 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testModuleShardLeaderMovement 09:09:59,842 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:59,842 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 09:09:59,842 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 09:09:59,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@2429a1d1} 09:09:59,844 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 09:09:59,844 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#916511707] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:59,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:59,844 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testModuleShardLeaderMovement 09:09:59,847 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 09:09:59,848 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 09:09:59,848 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement on shard member-1-shard-cars-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417] 09:09:59,848 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testModuleShardLeaderMovement 09:09:59,848 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 09:09:59,848 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 09:09:59,848 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement on shard member-1-shard-people-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-69069819] 09:09:59,848 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417] 09:09:59,848 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 09:09:59,849 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 09:09:59,850 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:59,851 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#1769879966] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:59,851 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#1769879966 created and ready for shard:member-1-shard-cars-config_testModuleShardLeaderMovement 09:09:59,851 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 09:09:59,852 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-69069819] 09:09:59,852 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-69069819] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:59,852 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Shard created, persistent : true 09:09:59,854 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#-902724563 created and ready for shard:member-1-shard-people-config_testModuleShardLeaderMovement 09:09:59,854 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-people-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 09:09:59,854 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#-902724563] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:59,854 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 09:09:59,855 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testModuleShardLeaderMovement: Recovery completed in in 3.734 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:59,856 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:09:59,856 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testModuleShardLeaderMovement is initialized 09:09:59,856 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:09:59,856 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:59,856 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 09:09:59,856 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-config_testModuleShardLeaderMovement: journal open: applyTo=0 09:09:59,858 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 09:09:59,858 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:59,858 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 09:09:59,858 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:09:59,858 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:59,858 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:09:59,858 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:59,858 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:09:59,859 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 09:09:59,859 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:09:59,859 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 09:09:59,859 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@444713f 09:09:59,859 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 09:09:59,861 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-config_testModuleShardLeaderMovement: Recovery completed in in 4.417 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:59,861 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:09:59,861 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:59,861 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from null to Follower 09:09:59,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:09:59,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testModuleShardLeaderMovement is initialized 09:09:59,864 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 09:09:59,864 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:59,864 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:09:59,864 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:59,864 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:09:59,864 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:59,864 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:09:59,864 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: became leader with frontend state for [] 09:09:59,864 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:09:59,864 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 09:09:59,864 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7de203a1 09:09:59,864 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testModuleShardLeaderMovement from Candidate to Leader 09:09:59,864 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:09:59,894 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#916511707] 09:09:59,895 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:09:59,895 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417] 09:09:59,895 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:59,900 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:09:59,905 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 09:09:59,906 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#876109116] 09:09:59,907 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#876109116] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:59,907 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:09:59,907 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:09:59,907 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:09:59,908 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testModuleShardLeaderMovement 09:09:59,908 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testModuleShardLeaderMovement 09:09:59,908 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:09:59,908 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 09:09:59,908 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 09:09:59,909 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@597d33b} 09:09:59,909 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 09:09:59,910 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:09:59,910 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 09:09:59,911 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 09:09:59,911 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 09:09:59,911 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement on shard member-1-shard-cars-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-102051581] 09:09:59,911 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testModuleShardLeaderMovement 09:09:59,912 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 09:09:59,912 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 09:09:59,912 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement on shard member-1-shard-people-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#490525106] 09:09:59,912 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-102051581] 09:09:59,912 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 09:09:59,912 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-102051581] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:59,913 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#1939532851 created and ready for shard:member-1-shard-cars-oper_testModuleShardLeaderMovement 09:09:59,913 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#490525106] 09:09:59,913 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 09:09:59,913 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#1939532851] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:59,913 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#490525106] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:59,913 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Shard created, persistent : true 09:09:59,914 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-people-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 09:09:59,914 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#-539677215 created and ready for shard:member-1-shard-people-oper_testModuleShardLeaderMovement 09:09:59,914 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#-539677215] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:09:59,915 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 09:09:59,915 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testModuleShardLeaderMovement: journal open: applyTo=0 09:09:59,917 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 09:09:59,918 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-oper_testModuleShardLeaderMovement: Recovery completed in in 2.403 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:59,918 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:09:59,918 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:59,918 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:09:59,918 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from null to Follower 09:09:59,918 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testModuleShardLeaderMovement is initialized 09:09:59,919 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 09:09:59,919 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed in in 4.066 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:09:59,919 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:09:59,919 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:59,919 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 09:09:59,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:09:59,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testModuleShardLeaderMovement is initialized 09:09:59,921 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 09:09:59,921 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:59,921 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:09:59,921 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:59,921 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:09:59,921 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:59,921 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:09:59,921 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 09:09:59,921 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:09:59,921 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 09:09:59,921 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 09:09:59,921 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:09:59,921 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:09:59,921 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:09:59,922 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:09:59,922 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:09:59,922 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:09:59,922 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 09:09:59,922 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:09:59,922 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 09:09:59,922 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@599edcd7 09:09:59,922 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testModuleShardLeaderMovement from Candidate to Leader 09:09:59,922 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@740df34d 09:09:59,922 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Candidate to Leader 09:09:59,922 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:09:59,961 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#876109116] 09:09:59,962 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:09:59,962 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-102051581] 09:09:59,980 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:09:59,997 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-1076081234775797387] 09:09:59,998 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.4.0] ... 09:10:00,2 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:00,2 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:10:00,4 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:00,4 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:00,20 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:00,23 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:00,30 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 09:10:00,42 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#619165501] 09:10:00,44 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 09:10:00,45 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:00,47 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:00,62 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 09:10:00,72 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#619165501] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:00,72 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:10:00,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 09:10:00,72 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 09:10:00,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@198b87d0} 09:10:00,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:00,73 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-42992392] 09:10:00,74 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:10:00,74 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 09:10:00,74 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 09:10:00,77 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-42992392] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:00,85 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 09:10:00,89 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@61b2a7c9} 09:10:00,89 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:00,103 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:10:00,114 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:00,115 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:00,115 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:00,115 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:00,115 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:10:00,121 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:00,141 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [7199861928892725127] 09:10:00,142 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.4.0] ... 09:10:00,146 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:00,146 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:10:00,150 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:00,151 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:00,170 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:00,172 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:00,175 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 09:10:00,180 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#141943981] 09:10:00,182 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:10:00,182 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 09:10:00,182 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 09:10:00,182 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#141943981] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:00,184 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@4a3f11ea} 09:10:00,184 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 09:10:00,185 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:00,185 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:00,187 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:00,191 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 09:10:00,192 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-251017842] 09:10:00,193 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-251017842] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:00,193 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:10:00,193 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 09:10:00,193 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 09:10:00,198 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@44f8e6dd} 09:10:00,198 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 09:10:00,198 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:00,224 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:10:00,230 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:00,232 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:00,232 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:00,232 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:00,232 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:10:00,832 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:10:00,832 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:10:00,834 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:00,834 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 09:10:00,834 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 09:10:00,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement 09:10:00,835 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 09:10:00,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:00,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:00,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 09:10:00,835 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:00,835 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement 09:10:00,835 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 09:10:00,835 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 09:10:00,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement 09:10:00,835 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 09:10:00,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:00,836 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement 09:10:00,836 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 09:10:00,836 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:10:00,836 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:00,836 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement 09:10:00,836 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 09:10:00,836 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement 09:10:00,836 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 09:10:00,836 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:10:00,847 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:00,847 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:00,847 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:00,847 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:00,847 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:00,847 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:10:00,847 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:00,847 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:10:01,46 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:01,46 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,46 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:01,46 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,46 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:01,47 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:10:01,47 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:01,47 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:10:01,104 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 09:10:01,104 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 09:10:01,105 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:01,105 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement] 09:10:01,110 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:01,110 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417, primaryVersion=13] 09:10:01,113 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417 for shard member-2-shard-cars-config_testModuleShardLeaderMovement 09:10:01,113 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-722880875] 09:10:01,114 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 09:10:01,115 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-722880875] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,115 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 09:10:01,115 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#736482625] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,117 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#736482625 created and ready for shard:member-2-shard-cars-config_testModuleShardLeaderMovement 09:10:01,117 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 09:10:01,117 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testModuleShardLeaderMovement 09:10:01,118 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 09:10:01,118 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:10:01,119 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-2-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 09:10:01,120 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-2-shard-cars-config_testModuleShardLeaderMovement: Recovery completed in in 3.028 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:01,120 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:10:01,121 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testModuleShardLeaderMovement is initialized 09:10:01,121 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:10:01,121 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,121 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 09:10:01,121 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 09:10:01,121 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from null to Follower 09:10:01,121 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2026-03-18T09:10:01.118199340Z 09:10:01,122 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 09:10:01,125 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 09:10:01,125 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:10:01,125 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:10:01,126 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2a011350 09:10:01,126 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,128 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-2-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2026-03-18T09:10:01.126326619Z 09:10:01,129 AM [cluster-test-shard-dispatcher-11] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 09:10:01,129 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:10:01,129 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1857809081] with name member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 09:10:01,129 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 627.1 μs 09:10:01,129 AM [cluster-test-shard-dispatcher-11] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 09:10:01,130 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1857809081] 09:10:01,131 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 09:10:01,131 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#-1857809081] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,132 AM [cluster-test-shard-dispatcher-27] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:10:01,131 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 103.8 μs 09:10:01,136 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417 09:10:01,136 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 09:10:01,137 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testModuleShardLeaderMovement 09:10:01,137 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 09:10:01,137 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:01,140 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6f75558f 09:10:01,141 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,141 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 09:10:01,141 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:10:01,142 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 09:10:01,142 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,142 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#619165501] 09:10:01,143 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:01,143 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-722880875] 09:10:01,143 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-722880875] 09:10:01,143 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-42992392] 09:10:01,144 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:10:01,144 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 09:10:01,144 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 09:10:01,144 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 09:10:01,144 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:01,145 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement] 09:10:01,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:01,147 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-102051581, primaryVersion=13] 09:10:01,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-102051581 for shard member-2-shard-cars-oper_testModuleShardLeaderMovement 09:10:01,149 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#1612561059] 09:10:01,149 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 09:10:01,149 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#1612561059] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,150 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#-1838977320 created and ready for shard:member-2-shard-cars-oper_testModuleShardLeaderMovement 09:10:01,150 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#-1838977320] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,150 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 09:10:01,152 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testModuleShardLeaderMovement 09:10:01,152 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 09:10:01,152 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:10:01,153 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 09:10:01,156 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2026-03-18T09:10:01.152850908Z 09:10:01,156 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-2-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 09:10:01,157 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-2-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed in in 3.637 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:01,157 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:10:01,157 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testModuleShardLeaderMovement is initialized 09:10:01,157 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:10:01,157 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,157 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 09:10:01,157 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 09:10:01,160 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 09:10:01,160 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 09:10:01,160 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 09:10:01,160 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:10:01,160 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e485fb4 09:10:01,160 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 09:10:01,160 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:10:01,163 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-2-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2026-03-18T09:10:01.161096043Z 09:10:01,163 AM [cluster-test-shard-dispatcher-11] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 09:10:01,163 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:10:01,164 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#1326842827] with name member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 09:10:01,164 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#1326842827] 09:10:01,164 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 560.8 μs 09:10:01,164 AM [cluster-test-shard-dispatcher-11] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 09:10:01,164 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 09:10:01,164 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#1326842827] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,164 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 167.4 μs 09:10:01,165 AM [cluster-test-shard-dispatcher-27] [INFO] AbstractLeader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:10:01,168 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-102051581 09:10:01,168 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 09:10:01,169 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testModuleShardLeaderMovement 09:10:01,169 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 09:10:01,170 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:01,170 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:10:01,173 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 09:10:01,173 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,175 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2d98ceb5 09:10:01,175 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 09:10:01,176 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-42992392] 09:10:01,176 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:10:01,176 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 09:10:01,176 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:10:01,177 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#1612561059] 09:10:01,177 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#1612561059] 09:10:01,178 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 09:10:01,178 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 09:10:01,179 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:01,179 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement] 09:10:01,181 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:01,181 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417, primaryVersion=13] 09:10:01,185 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,185 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417 for shard member-3-shard-cars-config_testModuleShardLeaderMovement 09:10:01,186 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 09:10:01,186 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,187 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 09:10:01,187 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-297490151 created and ready for shard:member-3-shard-cars-config_testModuleShardLeaderMovement 09:10:01,187 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#-297490151] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,188 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 09:10:01,190 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testModuleShardLeaderMovement 09:10:01,190 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 09:10:01,190 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:10:01,191 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 09:10:01,193 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-cars-config_testModuleShardLeaderMovement: Recovery completed in in 3.781 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:01,193 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:10:01,193 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testModuleShardLeaderMovement is initialized 09:10:01,193 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:10:01,193 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,193 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 09:10:01,193 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]}}" message is greater than follower's term 0 - updating term 09:10:01,194 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2026-03-18T09:10:01.190956462Z 09:10:01,195 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 09:10:01,196 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 09:10:01,196 AM [cluster-test-shard-dispatcher-23] [INFO] SnapshotManager - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:10:01,196 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@40c77a9a 09:10:01,196 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from null to Follower 09:10:01,196 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,196 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:10:01,198 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-3-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2026-03-18T09:10:01.196577287Z 09:10:01,198 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 09:10:01,198 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:10:01,199 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#942692718] with name member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 09:10:01,199 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#942692718] 09:10:01,199 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 664.7 μs 09:10:01,199 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 09:10:01,199 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 09:10:01,199 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 91.23 μs 09:10:01,199 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#942692718] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,201 AM [cluster-test-shard-dispatcher-27] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 09:10:01,205 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1998eea6 09:10:01,205 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,205 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:10:01,205 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417 09:10:01,205 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 09:10:01,205 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:01,206 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testModuleShardLeaderMovement 09:10:01,206 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 09:10:01,208 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 09:10:01,209 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,209 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:10:01,209 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 09:10:01,209 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#141943981] 09:10:01,209 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:01,209 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,209 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,210 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-251017842] 09:10:01,210 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:10:01,210 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 09:10:01,210 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 09:10:01,210 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 09:10:01,211 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:01,211 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement] 09:10:01,213 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:01,213 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-102051581, primaryVersion=13] 09:10:01,216 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-102051581 for shard member-3-shard-cars-oper_testModuleShardLeaderMovement 09:10:01,216 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-1077263202] 09:10:01,216 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 09:10:01,217 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 09:10:01,219 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#-393335405 created and ready for shard:member-3-shard-cars-oper_testModuleShardLeaderMovement 09:10:01,219 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-1077263202] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,220 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#-393335405] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,221 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 09:10:01,221 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testModuleShardLeaderMovement 09:10:01,222 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 09:10:01,222 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:10:01,224 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-3-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 09:10:01,225 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-3-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed in in 3.368 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:01,225 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2026-03-18T09:10:01.222643546Z 09:10:01,225 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:10:01,225 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:10:01,226 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,226 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testModuleShardLeaderMovement is initialized 09:10:01,226 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 09:10:01,226 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]}}" message is greater than follower's term 0 - updating term 09:10:01,228 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 09:10:01,228 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:10:01,228 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 09:10:01,229 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 09:10:01,229 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@bb122d4 09:10:01,229 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 09:10:01,229 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:10:01,231 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-3-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2026-03-18T09:10:01.229380278Z 09:10:01,232 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 09:10:01,232 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:10:01,232 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1910369731] with name member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 09:10:01,232 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1910369731] 09:10:01,232 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 648.8 μs 09:10:01,232 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 09:10:01,233 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 09:10:01,233 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1910369731] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,233 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 115.3 μs 09:10:01,234 AM [cluster-test-shard-dispatcher-27] [INFO] AbstractLeader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 09:10:01,237 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#-102051581 09:10:01,238 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 09:10:01,238 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testModuleShardLeaderMovement 09:10:01,238 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 09:10:01,239 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:01,239 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 09:10:01,241 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@48028e19 09:10:01,241 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 09:10:01,241 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 09:10:01,241 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 09:10:01,241 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-251017842] 09:10:01,241 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:10:01,241 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done true 09:10:01,241 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 09:10:01,242 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-1077263202] 09:10:01,242 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,242 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#-1077263202] 09:10:01,242 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#916511707] 09:10:01,242 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:01,243 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417] 09:10:01,243 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417] 09:10:01,243 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#619165501] 09:10:01,243 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:01,243 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-722880875] 09:10:01,243 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-722880875] 09:10:01,244 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#141943981] 09:10:01,244 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:01,244 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,244 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,246 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-1 for shard ShardName{value=cars}, datastoreType Config 09:10:01,247 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#916511707] 09:10:01,247 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:01,247 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417] 09:10:01,248 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417] 09:10:01,248 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 09:10:01,249 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 09:10:01,251 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#916511707] 09:10:01,252 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:01,252 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417] 09:10:01,252 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417] 09:10:01,260 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#619165501] 09:10:01,260 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:01,260 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-722880875] 09:10:01,260 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-722880875] 09:10:01,261 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#141943981] 09:10:01,261 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:01,261 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,261 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,262 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-2 for shard ShardName{value=cars}, datastoreType Config 09:10:01,262 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#619165501] 09:10:01,262 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:01,262 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-722880875] 09:10:01,262 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-722880875] 09:10:01,262 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 09:10:01,271 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5df286ee 09:10:01,271 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@799f27db 09:10:01,271 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@799f27db 09:10:01,271 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 09:10:01,274 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@19f8b286 09:10:01,274 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3218aaff 09:10:01,274 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:10:01,274 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:10:01,276 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 09:10:01,278 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:10:01,283 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 2 09:10:01,283 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:10:01,283 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:10:01,283 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,283 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:10:01,283 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@59a52db9 09:10:01,283 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 09:10:01,285 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 2 in "RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:01,286 AM [cluster-test-shard-dispatcher-27] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 2 in "RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 1 - switching to Follower 09:10:01,291 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 09:10:01,291 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:10:01,291 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 09:10:01,291 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 2 09:10:01,291 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@33e4032 09:10:01,291 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:10:01,291 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:10:01,291 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,291 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 09:10:01,291 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@606ca614 09:10:01,291 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 09:10:01,291 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,291 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 09:10:01,293 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 09:10:01,293 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@53356320 09:10:01,293 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 21.65 ms 09:10:01,293 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:10:01,293 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,293 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3257af90 09:10:01,293 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:10:01,293 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,298 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:10:01,298 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:10:01,301 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 09:10:01,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#619165501] 09:10:01,301 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:01,301 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-722880875] 09:10:01,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-722880875] 09:10:01,302 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#916511707] 09:10:01,303 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:01,303 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417] 09:10:01,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#-213018417] 09:10:01,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#141943981] 09:10:01,303 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:01,303 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,304 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-3 for shard ShardName{value=cars}, datastoreType Config 09:10:01,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#141943981] 09:10:01,304 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:01,304 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,304 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,304 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 09:10:01,307 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5fe33730 09:10:01,307 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5316619c 09:10:01,307 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5316619c 09:10:01,307 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 09:10:01,309 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@35d12da5 09:10:01,309 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@570aed9e 09:10:01,310 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 09:10:01,315 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 3 09:10:01,315 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 3 09:10:01,315 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:10:01,315 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,315 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:10:01,315 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@19da42ab 09:10:01,316 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 09:10:01,317 AM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 3 in "RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 2 - switching to Follower 09:10:01,317 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 3 in "RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 2 - updating term 09:10:01,321 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 3 09:10:01,321 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:10:01,321 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,322 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 09:10:01,322 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@33228daa 09:10:01,322 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 09:10:01,324 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:01,324 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:10:01,324 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 09:10:01,324 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:10:01,324 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@45eaaf94 09:10:01,324 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 09:10:01,324 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,325 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-3-shard-cars-config_testModuleShardLeaderMovement 09:10:01,325 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:10:01,325 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5cd7e6bf 09:10:01,325 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-3-shard-cars-config_testModuleShardLeaderMovement in 18.58 ms 09:10:01,326 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,326 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@599de6cb 09:10:01,326 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,326 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:10:01,327 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 09:10:01,327 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#141943981] 09:10:01,327 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:01,327 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,327 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#2051537858] 09:10:01,328 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 09:10:01,329 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testModuleShardLeaderMovement 09:10:01,329 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 09:10:01,329 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:10:01,329 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4a4ea396 09:10:01,330 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1741910c 09:10:01,330 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1741910c 09:10:01,330 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 09:10:01,330 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:10:01,331 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 09:10:01,332 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1208691b 09:10:01,335 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4b89c87c 09:10:01,338 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 4 09:10:01,338 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:10:01,338 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:10:01,338 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,338 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 09:10:01,338 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5b845842 09:10:01,338 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 09:10:01,340 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 3 - updating term 09:10:01,340 AM [cluster-test-shard-dispatcher-24] [INFO] AbstractLeader - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 3 - switching to Follower 09:10:01,345 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 4 09:10:01,345 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 09:10:01,345 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:01,346 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 09:10:01,346 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@19867d94 09:10:01,346 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 09:10:01,347 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:10:01,347 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:10:01,347 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 09:10:01,347 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 09:10:01,347 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@23501c71 09:10:01,348 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 09:10:01,348 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,349 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-1-shard-cars-config_testModuleShardLeaderMovement 09:10:01,349 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@76ce037f 09:10:01,349 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-1-shard-cars-config_testModuleShardLeaderMovement in 19.44 ms 09:10:01,349 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,349 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:10:01,350 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 09:10:01,350 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3cb1cca0 09:10:01,350 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 09:10:01,350 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 09:10:01,350 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testModuleShardLeaderMovement: journal closed 09:10:01,351 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:10:01,351 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:10:01,352 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 09:10:01,352 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testModuleShardLeaderMovement 09:10:01,352 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 09:10:01,353 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#141943981] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#141943981] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,353 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f52cc8403d0] to Actor[pekko://cluster-test/user/$a#-895661344] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#-895661344] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,354 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 09:10:01,354 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 09:10:01,355 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testModuleShardLeaderMovement: journal closed 09:10:01,355 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:10:01,356 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:10:01,357 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:01,357 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/18/2026 09:10:01.364] [cluster-test-pekko.remote.default-remote-dispatcher-6] [org.apache.pekko.stream.Log(pekko://cluster-test/system/Materializers/StreamSupervisor-25)] [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/18/2026 09:10:01.364] [cluster-test-pekko.remote.default-remote-dispatcher-6] [org.apache.pekko.stream.Log(pekko://cluster-test/system/Materializers/StreamSupervisor-25)] [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/18/2026 09:10:01.365] [cluster-test-pekko.remote.default-remote-dispatcher-6] [org.apache.pekko.stream.Log(pekko://cluster-test/system/Materializers/StreamSupervisor-25)] [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/18/2026 09:10:01.365] [cluster-test-pekko.remote.default-remote-dispatcher-6] [org.apache.pekko.stream.Log(pekko://cluster-test/system/Materializers/StreamSupervisor-25)] [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:01,391 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 09:10:01,392 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testModuleShardLeaderMovement 09:10:01,392 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 09:10:01,393 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 09:10:01,394 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testModuleShardLeaderMovement: journal closed 09:10:01,394 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:10:01,395 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:10:01,395 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 09:10:01,395 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#619165501] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#619165501] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,396 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testModuleShardLeaderMovement 09:10:01,396 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f52cc8403d0] to Actor[pekko://cluster-test/user/$a#-165490144] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#-165490144] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:01,396 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 09:10:01,397 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 09:10:01,397 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testModuleShardLeaderMovement: journal closed 09:10:01,398 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:10:01,398 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:10:01,400 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:01,400 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:01,852 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:10:02,371 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [7199861928892725127] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:02,416 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:02,417 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:02,419 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:10:02,423 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-1076081234775797387] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/18/2026 09:10:02.423] [cluster-test-pekko.actor.internal-dispatcher-5] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:02,432 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 09:10:02,433 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testModuleShardLeaderMovement 09:10:02,433 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testModuleShardLeaderMovement 09:10:02,433 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:10:02,433 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@7256d6dd 09:10:02,433 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@7256d6dd 09:10:02,433 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@23367398 09:10:02,434 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@638cdd83 09:10:02,434 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@638cdd83 09:10:02,434 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 09:10:02,434 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Stopping Shard 09:10:02,434 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-1-shard-people-config_testModuleShardLeaderMovement: journal closed 09:10:06,952 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 09:10:06,952 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 09:10:06,954 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:06,954 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:06,954 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:06,954 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:07,971 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:10:09,29 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:10:10,3 AM [cluster-test-pekko.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:10:11,580 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer aborted 09:10:11,580 AM [cluster-test-shard-dispatcher-27] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Failed to transfer leadership in 9.147 s 09:10:11,581 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In unpauseLeader 09:10:11,581 AM [cluster-test-shard-dispatcher-27] [WARN] Leader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 09:10:11,582 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 09:10:11,582 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:10:11,582 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 09:10:11,583 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 09:10:11,583 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 09:10:11,584 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testModuleShardLeaderMovement: journal closed 09:10:11,585 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:10:11,585 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:10:11,586 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 09:10:11,586 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testModuleShardLeaderMovement 09:10:11,586 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testModuleShardLeaderMovement 09:10:11,587 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:10:11,587 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@1349f678 09:10:11,587 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7c9126e7 09:10:11,587 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@fcc70cd 09:10:11,587 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@1349f678 09:10:11,587 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@fcc70cd 09:10:11,587 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 09:10:11,587 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Stopping Shard 09:10:11,588 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testModuleShardLeaderMovement: journal closed 09:10:15,230 AM [cluster-test-shard-dispatcher-13] [WARN] Leader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 09:10:15,231 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:10:15,231 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer aborted 09:10:15,231 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: Failed to transfer leadership in 3.644 s 09:10:15,231 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In unpauseLeader 09:10:15,232 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 09:10:15,232 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 09:10:15,232 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 09:10:15,232 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 09:10:15,233 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testModuleShardLeaderMovement: journal closed 09:10:15,233 AM [cluster-test-pekko.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:10:15,233 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:10:15,235 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:15,235 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:15,239 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:10:15.243] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:15,272 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:15,293 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-633178522890159645] 09:10:15,294 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:10:15,298 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:10:15,298 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:10:15,301 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:15,302 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:15,317 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:15,323 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:10:15,323 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:10:15,323 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:10:15,326 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:15,330 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 09:10:15,338 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1563279735] 09:10:15,338 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1563279735] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,342 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} started 09:10:15,343 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:10:15,343 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:10:15,343 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:10:15,343 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:15,344 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardReplica 09:10:15,344 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardReplica 09:10:15,345 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:15,345 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:10:15,345 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:10:15,345 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@7a85f7f4} 09:10:15,345 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:15,346 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:15,347 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardReplica 09:10:15,348 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:10:15,348 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:10:15,348 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica on shard member-1-shard-cars-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1057930693] 09:10:15,348 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardReplica 09:10:15,349 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 09:10:15,350 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:10:15,350 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:10:15,350 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica on shard member-1-shard-people-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#-1751687180] 09:10:15,350 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1057930693] 09:10:15,350 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 09:10:15,351 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1057930693] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,351 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:15,351 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#863290520 created and ready for shard:member-1-shard-cars-config_testRemoveShardReplica 09:10:15,351 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#863290520] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,355 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 09:10:15,356 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#-1751687180] 09:10:15,356 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 09:10:15,356 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#-1751687180] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,357 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#2006518454 created and ready for shard:member-1-shard-people-config_testRemoveShardReplica 09:10:15,357 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#2006518454] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,357 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:15,359 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-410225813] 09:10:15,359 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-410225813] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,361 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:10:15,361 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:10:15,361 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:10:15,361 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardReplica 09:10:15,362 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardReplica 09:10:15,362 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:15,362 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:10:15,362 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:10:15,362 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:15,363 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 09:10:15,363 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testRemoveShardReplica: Recovery completed in in 7.894 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:15,364 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@567b58e0} 09:10:15,364 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:15,364 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardReplica 09:10:15,364 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,364 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,364 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:10:15,364 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardReplica is initialized 09:10:15,364 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:10:15,364 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:10:15,364 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-1-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#1919663318] 09:10:15,364 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardReplica 09:10:15,365 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#1919663318] 09:10:15,365 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 09:10:15,365 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#1919663318] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,365 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:10:15,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#-578824203] 09:10:15,365 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 09:10:15,365 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 09:10:15,366 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:15,367 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 09:10:15,368 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#-2003293905 created and ready for shard:member-1-shard-people-oper_testRemoveShardReplica 09:10:15,368 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#-578824203] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,368 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 09:10:15,370 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#-53110233 created and ready for shard:member-1-shard-cars-oper_testRemoveShardReplica 09:10:15,370 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:15,372 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 09:10:15,373 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:10:15,373 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:10:15,373 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-1-shard-people-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#-578824203] 09:10:15,374 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:15,374 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-1-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:15,374 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:15,374 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-1-shard-cars-oper_testRemoveShardReplica: Recovery completed in in 2.382 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:15,374 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-config_testRemoveShardReplica: Recovery completed in in 6.085 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:15,375 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:10:15,375 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardReplica is initialized 09:10:15,375 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:10:15,375 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,375 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,375 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,375 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardReplica is initialized 09:10:15,375 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,375 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 09:10:15,375 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from null to Follower 09:10:15,375 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:10:15,375 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-oper_testRemoveShardReplica: Recovery completed in in 7.666 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:15,375 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,375 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from null to Follower 09:10:15,375 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,375 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:10:15,375 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 09:10:15,375 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardReplica is initialized 09:10:15,375 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:10:15,376 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from null to Follower 09:10:15,377 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from null to Follower 09:10:15,377 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#-2003293905] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,377 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#-53110233] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,387 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:10:15,387 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from null to Follower 09:10:15,388 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:10:15,405 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-5233942713144746308] 09:10:15,406 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.4.0] ... 09:10:15,409 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:15,409 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:10:15,411 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:15,411 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:15,423 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:15,427 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:15,430 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 09:10:15,433 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#198281141] 09:10:15,434 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:10:15,434 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 09:10:15,434 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 09:10:15,434 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 09:10:15,434 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 09:10:15,435 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#198281141] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,438 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} started 09:10:15,438 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@57dce759} 09:10:15,439 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:15,439 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 09:10:15,440 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:15,447 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:15,450 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 09:10:15,451 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 09:10:15,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1619026162] 09:10:15,452 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 09:10:15,452 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1619026162] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,452 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#903578939 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 09:10:15,452 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:15,452 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#547410267] 09:10:15,452 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#903578939] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,453 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#547410267] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,453 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 09:10:15,453 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#426897157 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 09:10:15,453 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:15,453 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#426897157] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,454 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 09:10:15,455 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1203569297] 09:10:15,456 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1203569297] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,456 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:10:15,456 AM [cluster-test-shard-dispatcher-22] [INFO] Recovery - member-2-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:15,457 AM [cluster-test-shard-dispatcher-22] [INFO] Recovery - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed in in 2.720 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:15,457 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 09:10:15,457 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 09:10:15,457 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 09:10:15,457 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 09:10:15,459 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:10:15,459 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 09:10:15,462 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 09:10:15,462 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@720aa521} 09:10:15,462 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:15,462 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 09:10:15,463 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 09:10:15,463 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 09:10:15,463 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#-897295055] 09:10:15,464 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 09:10:15,464 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#1735659864] 09:10:15,464 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 09:10:15,464 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#872481108 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 09:10:15,464 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:15,464 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#1000858567 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 09:10:15,465 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:15,465 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 09:10:15,466 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 09:10:15,467 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:15,467 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-people-config_testRemoveShardReplica: Recovery completed in in 3.705 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:15,467 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-2-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:15,468 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-2-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:15,468 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed in in 2.264 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:15,469 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,469 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,469 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 09:10:15,469 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:10:15,469 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 09:10:15,469 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:10:15,469 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 09:10:15,469 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:10:15,471 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 09:10:15,472 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,472 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,472 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 09:10:15,472 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:10:15,472 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 09:10:15,472 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed in in 6.838 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:15,473 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,473 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,473 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 09:10:15,473 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:10:15,473 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 09:10:15,473 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:10:15,473 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 09:10:15,480 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#-897295055] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,480 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#1735659864] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,480 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#872481108] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,480 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#1000858567] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,484 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,484 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,484 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 09:10:15,484 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:10:15,484 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 09:10:15,491 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:15,504 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:10:15,512 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:15,513 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:15,513 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:15,513 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:10:15,513 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:10:15,513 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:10:15,513 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:10:15,513 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:10:15,514 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:10:15,514 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:10:15,514 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:10:15,514 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:10:15,514 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:10:15,514 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:10:15,514 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:10:15,520 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [1300322287080634739] 09:10:15,521 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.4.0] ... 09:10:15,523 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:15,523 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:10:15,534 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:15,534 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:15,541 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:15,543 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:15,545 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 09:10:15,551 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1664233646] 09:10:15,553 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:10:15,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 09:10:15,554 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 09:10:15,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardReplica 09:10:15,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardReplica 09:10:15,555 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1664233646] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,555 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} started 09:10:15,556 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:15,556 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@64f231b8} 09:10:15,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:15,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardReplica 09:10:15,558 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:15,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardReplica 09:10:15,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-72373365] 09:10:15,561 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 09:10:15,561 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:15,562 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 09:10:15,563 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-72373365] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,564 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#594187256 created and ready for shard:member-3-shard-cars-config_testRemoveShardReplica 09:10:15,564 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#594187256] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,564 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 09:10:15,565 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-3-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:15,565 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-3-shard-cars-config_testRemoveShardReplica: Recovery completed in in 2.862 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:15,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:10:15,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardReplica is initialized 09:10:15,568 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 09:10:15,574 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica#-1771029350] 09:10:15,574 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 09:10:15,575 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1433930437] 09:10:15,575 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica#-1771029350] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,576 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#1824301398] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,576 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 09:10:15,576 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:10:15,576 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:10:15,576 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 09:10:15,576 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 09:10:15,577 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardReplica 09:10:15,577 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardReplica 09:10:15,577 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@2c1ce32e} 09:10:15,577 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:10:15,577 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:15,577 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardReplica 09:10:15,578 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:10:15,578 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardReplica 09:10:15,578 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#743447992] 09:10:15,578 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 09:10:15,579 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#1146342969 created and ready for shard:member-3-shard-cars-oper_testRemoveShardReplica 09:10:15,579 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:15,579 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:15,580 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 09:10:15,583 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-3-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:15,583 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardReplica from null to Follower 09:10:15,583 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-3-shard-cars-oper_testRemoveShardReplica: Recovery completed in in 2.528 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:15,583 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#-1977710678] 09:10:15,584 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 09:10:15,584 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,584 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,584 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 09:10:15,585 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#-2037042744 created and ready for shard:member-3-shard-people-oper_testRemoveShardReplica 09:10:15,585 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-3-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:15,586 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 09:10:15,587 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#1824301398 created and ready for shard:member-3-shard-people-config_testRemoveShardReplica 09:10:15,588 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-3-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:15,588 AM [cluster-test-shard-dispatcher-25] [INFO] Recovery - member-3-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:15,589 AM [cluster-test-shard-dispatcher-25] [INFO] Recovery - member-3-shard-people-oper_testRemoveShardReplica: Recovery completed in in 2.741 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:15,589 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,589 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,589 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 09:10:15,590 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1433930437] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,590 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#743447992] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,590 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:10:15,590 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#1146342969] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,590 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardReplica is initialized 09:10:15,590 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#-1977710678] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,590 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:10:15,590 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardReplica is initialized 09:10:15,590 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:10:15,590 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#-2037042744] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:15,590 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardReplica from null to Follower 09:10:15,591 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 09:10:15,592 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:10:15,592 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardReplica from null to Follower 09:10:15,596 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-3-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:15,599 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-3-shard-people-config_testRemoveShardReplica: Recovery completed in in 5.006 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:15,599 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,599 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,599 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , received role change from null to Follower 09:10:15,599 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:15,599 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:10:15,599 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:10:15,599 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:10:15,599 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:10:15,599 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:10:15,600 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardReplica is initialized 09:10:15,600 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:10:15,600 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:10:15,600 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:10:15,600 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:15,600 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:10:15,600 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:10:15,600 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:10:15,600 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:10:15,600 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:10:15,600 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:10:15,601 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardReplica from null to Follower 09:10:15,730 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 1 09:10:15,730 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:15,730 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,730 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,730 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Follower to Candidate 09:10:15,730 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Follower to Candidate 09:10:15,750 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 1 09:10:15,750 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:15,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,750 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,750 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Follower to Candidate 09:10:15,751 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Follower to Candidate 09:10:15,791 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 1 09:10:15,792 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:15,792 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,792 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,792 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Follower to Candidate 09:10:15,792 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Follower to Candidate 09:10:15,792 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 1 09:10:15,792 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:15,792 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:15,792 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:15,792 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Follower to Candidate 09:10:15,793 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Follower to Candidate 09:10:16,150 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 2 09:10:16,150 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 2 09:10:16,151 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 2 09:10:16,160 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 2 09:10:16,319 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:10:16,320 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:10:16,320 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:16,320 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:16,320 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:16,320 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:16,320 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:10:16,320 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:10:16,321 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:10:16,321 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:10:16,321 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:10:16,321 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:10:16,321 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:10:16,321 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:10:16,321 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:10:16,321 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:10:16,321 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:10:16,321 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardReplica: Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:16,330 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:16,330 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:16,330 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:16,330 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:10:16,330 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:10:16,443 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardReplica: Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:10:16,444 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:10:16,490 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 3 09:10:16,493 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:16,494 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:16,500 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:16,500 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:16,500 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: became leader with frontend state for [] 09:10:16,501 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Candidate to Leader 09:10:16,501 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6c3e5560 09:10:16,501 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Candidate to Leader 09:10:16,502 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1660dd25 09:10:16,502 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 09:10:16,502 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done false 09:10:16,502 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@46474b2d 09:10:16,509 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 3 09:10:16,509 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 3 09:10:16,511 AM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:16,511 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:16,511 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:16,512 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:16,518 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:16,518 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:16,518 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:16,518 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:16,518 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: became leader with frontend state for [] 09:10:16,518 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Candidate to Leader 09:10:16,518 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: became leader with frontend state for [] 09:10:16,518 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 3 09:10:16,518 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3af70b27 09:10:16,519 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@ee1e294 09:10:16,519 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Candidate to Leader 09:10:16,519 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Candidate to Leader 09:10:16,519 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Candidate to Leader 09:10:16,519 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:10:16,521 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done false 09:10:16,521 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@f319bc6 09:10:16,521 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 09:10:16,521 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 09:10:16,521 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done false 09:10:16,521 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:16,521 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-3-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:16,521 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3e28ac1 09:10:16,521 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4c1f78e0 09:10:16,521 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@cced24d 09:10:16,522 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:10:16,522 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:10:16,529 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:16,529 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:16,529 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: became leader with frontend state for [] 09:10:16,529 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Candidate to Leader 09:10:16,529 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@143463f4 09:10:16,529 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Candidate to Leader 09:10:16,529 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 09:10:16,530 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 09:10:16,530 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:10:16,531 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7943233a 09:10:16,531 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardReplica is ready 09:10:16,531 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardReplica status sync done false 09:10:16,531 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 09:10:16,531 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@141413ae 09:10:16,531 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 09:10:16,531 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1563279735] 09:10:16,532 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:16,532 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 09:10:16,532 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1057930693] 09:10:16,532 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1057930693] 09:10:16,534 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#198281141] 09:10:16,534 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:16,534 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1619026162] 09:10:16,534 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1619026162] 09:10:16,535 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1664233646] 09:10:16,535 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:16,535 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-72373365] 09:10:16,535 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-72373365] 09:10:16,538 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-3}, datastoreType Config 09:10:16,538 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 09:10:16,540 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:16,541 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, primaryVersion=13] 09:10:16,542 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica for shard member-3-shard-cars-config_testRemoveShardReplica 09:10:16,543 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:10:16,552 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 09:10:16,552 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 09:10:16,553 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 09:10:16,553 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:16,555 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 09:10:16,555 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1563279735] 09:10:16,555 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:16,555 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1057930693] 09:10:16,556 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1057930693] 09:10:16,558 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-72373365] with 27000 ms timeout 09:10:16,558 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Stopping Shard 09:10:16,559 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardReplica: journal closed 09:10:16,559 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#198281141] 09:10:16,560 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:16,560 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1619026162] 09:10:16,560 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 09:10:16,560 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [people] 09:10:16,560 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-72373365] 09:10:16,561 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1619026162] 09:10:16,561 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1664233646] 09:10:16,562 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:16,562 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [people] 09:10:16,563 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:10:16,564 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2559] as [Down] 09:10:16,568 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardReplica 09:10:16,568 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 09:10:16,568 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:10:16,568 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Stopping Shard 09:10:16,569 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal closed 09:10:16,569 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Stopping Shard 09:10:16,569 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal closed 09:10:16,569 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:10:16,569 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:10:16,569 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 09:10:16,570 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 09:10:16,570 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 09:10:16,570 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 09:10:16,570 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:10:16,570 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Stopping Shard 09:10:16,570 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Stopping Shard 09:10:16,571 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal closed 09:10:16,571 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal closed 09:10:16,571 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:10:16,571 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:10:16,572 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Node has been marked as DOWN. Shutting down myself 09:10:16,572 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:16,572 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:16,576 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Shutting down... 09:10:16,576 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Successfully shut down 09:10:16,578 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:16,578 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:16,579 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:16,579 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:16,581 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:10:16,624 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:16,638 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-8507291740446489021] 09:10:16,638 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.4.0] ... 09:10:16,639 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Registered cluster JMX MBean [pekko:type=Cluster] 09:10:16,639 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:10:16,643 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:16,643 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:16,663 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:16,674 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:16,678 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 09:10:16,684 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-632194329] 09:10:16,685 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-632194329] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:16,685 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:10:16,685 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 09:10:16,685 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 09:10:16,686 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 09:10:16,686 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 09:10:16,686 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@1e9b6e82} 09:10:16,686 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} started 09:10:16,687 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:16,688 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:16,688 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 09:10:16,689 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:16,692 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 09:10:16,694 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 09:10:16,695 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1132474666] 09:10:16,695 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 09:10:16,696 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:16,696 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#-456063283] 09:10:16,697 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 09:10:16,697 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-2083627167 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 09:10:16,697 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:16,698 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 09:10:16,699 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#1268883682 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 09:10:16,699 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} started 09:10:16,699 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 09:10:16,700 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-2-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:16,701 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 09:10:16,702 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-2-shard-people-config_testRemoveShardReplica: Recovery completed in in 2.788 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:16,703 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-2-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:16,703 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1132474666] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:16,703 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#-456063283] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:16,703 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-2083627167] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:16,703 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#1268883682] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:16,704 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed in in 2.595 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:16,704 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:16,704 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:16,704 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 09:10:16,704 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:10:16,704 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 09:10:16,704 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:10:16,704 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 09:10:16,704 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:16,704 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:16,704 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:10:16,704 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 09:10:16,704 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 09:10:16,705 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#2084100676] 09:10:16,705 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#2084100676] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:16,706 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:10:16,707 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 09:10:16,707 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 09:10:16,707 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 09:10:16,707 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 09:10:16,707 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 09:10:16,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 09:10:16,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@66ac0ee5} 09:10:16,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:16,708 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 09:10:16,710 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 09:10:16,710 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#618300554] 09:10:16,710 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 09:10:16,710 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#618300554] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:16,710 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#-1853288298 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 09:10:16,710 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:16,711 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#-1853288298] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:16,712 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 09:10:16,712 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#-1928110742] 09:10:16,712 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 09:10:16,712 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#-1928110742] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:16,713 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 09:10:16,713 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#1235540360 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 09:10:16,713 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#1235540360] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:16,714 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 09:10:16,716 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-2-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:16,716 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed in in 2.454 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:16,716 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-2-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:16,716 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:16,716 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:16,717 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed in in 4.789 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:16,717 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 09:10:16,717 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:10:16,717 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 09:10:16,717 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:16,717 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:16,717 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 09:10:16,717 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:10:16,717 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:10:16,717 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 09:10:16,717 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 09:10:16,717 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 09:10:16,717 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 09:10:16,821 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done true 09:10:16,823 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:16,840 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done true 09:10:16,840 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done true 09:10:16,841 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:16,841 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:17,144 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:17,162 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:17,162 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:17,339 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:10:17,461 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:10:17,481 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:17,481 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:17,590 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-5233942713144746308] but current is [-8507291740446489021]. Reason: ActorSystem terminated 09:10:17,592 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-5233942713144746308] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:17,617 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:10:17,632 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#-489676075]] intended for system uid [-5233942713144746308], self uid is [-8507291740446489021] 09:10:17,657 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:10:17,694 AM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-people-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 09:10:17,695 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#1334229881]] intended for system uid [-5233942713144746308], self uid is [-8507291740446489021] 09:10:17,696 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 09:10:17,696 AM [cluster-test-shard-dispatcher-16] [INFO] Follower - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-people-oper_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 09:10:17,696 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=0, prevLogTerm=3, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 09:10:17,699 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2d38992e 09:10:17,700 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): The followers log is empty and the senders prevLogIndex is 0 09:10:17,700 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@414d20bd 09:10:17,700 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 09:10:17,700 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done true 09:10:17,700 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 09:10:17,700 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 09:10:17,700 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done true 09:10:17,700 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): The followers log is empty and the senders prevLogIndex is 0 09:10:17,700 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 09:10:17,700 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@787b1d14 09:10:17,700 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 09:10:17,700 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done true 09:10:17,701 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@58463c38 09:10:17,701 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 09:10:17,701 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 09:10:17,703 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1199, lastApplied : -1, commitIndex : -1 09:10:17,703 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1180, lastApplied : -1, commitIndex : -1 09:10:17,703 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1180, lastApplied : -1, commitIndex : -1 09:10:17,703 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1151, lastApplied : 0, commitIndex : 0 09:10:17,703 AM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 09:10:17,703 AM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): follower member-2-shard-cars-config_testRemoveShardReplica appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 09:10:17,704 AM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 09:10:17,704 AM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): follower member-2-shard-cars-config_testRemoveShardReplica appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 09:10:17,706 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:10:17,781 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:17,801 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:17,801 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:18,101 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:10:18,102 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 399, lastApplied : -1, commitIndex : -1 09:10:18,104 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 323, lastApplied : -1, commitIndex : -1 09:10:18,119 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:10:18,119 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:10:18,120 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:18,120 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:18,121 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 417, lastApplied : -1, commitIndex : -1 09:10:18,121 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 417, lastApplied : -1, commitIndex : -1 09:10:18,130 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 422, lastApplied : 0, commitIndex : 0 09:10:18,421 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 09:10:18,421 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:18,440 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:18,440 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:18,441 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:18,441 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:18,450 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 09:10:18,677 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:10:18,740 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:18,741 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:18,762 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:18,762 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:18,762 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:18,762 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:18,770 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 09:10:19,60 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:19,60 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:19,81 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:19,82 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:19,82 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:19,82 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:19,90 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 09:10:19,360 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#1334229881]] intended for system uid [-5233942713144746308], self uid is [-8507291740446489021] 09:10:19,382 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:19,382 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:19,402 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:19,402 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:19,403 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:19,403 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:19,410 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 09:10:19,585 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#-489676075]] intended for system uid [-5233942713144746308], self uid is [-8507291740446489021] 09:10:19,697 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:10:19,702 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:19,702 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:19,721 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:19,721 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:19,721 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:19,722 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:19,731 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 09:10:20,21 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:20,22 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:20,42 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:20,42 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:20,43 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:20,43 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:20,51 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 09:10:20,342 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:20,342 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:20,362 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:20,363 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:20,363 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:20,364 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:20,370 AM [cluster-test-shard-dispatcher-17] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 09:10:20,662 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:20,662 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:20,682 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:20,683 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:20,683 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:20,683 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:20,691 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 09:10:20,719 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:10:20,981 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:20,982 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:21,2 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:21,2 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:21,3 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:21,3 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:21,12 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 09:10:21,301 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:21,301 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:21,320 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:10:21,320 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:10:21,320 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:21,320 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:21,330 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : 0, commitIndex : 0 09:10:21,380 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#1334229881]] intended for system uid [-5233942713144746308], self uid is [-8507291740446489021] 09:10:21,618 AM [cluster-test-shard-dispatcher-16] [WARN] Leader - member-1-shard-people-config_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 09:10:21,619 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:10:21,619 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:21,619 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 09:10:21,619 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 09:10:21,621 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:21,622 AM [cluster-test-shard-dispatcher-16] [INFO] IsolatedLeader - member-1-shard-people-config_testRemoveShardReplica: IsolatedLeader member-1-shard-people-config_testRemoveShardReplica switching from IsolatedLeader to Leader 09:10:21,622 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 09:10:21,622 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:21,622 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 09:10:21,622 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:21,622 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from IsolatedLeader to Leader 09:10:21,622 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 09:10:21,638 AM [cluster-test-shard-dispatcher-15] [WARN] Leader - member-1-shard-people-oper_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 09:10:21,638 AM [cluster-test-shard-dispatcher-16] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 09:10:21,639 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:10:21,639 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:10:21,639 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:21,639 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:21,639 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 09:10:21,639 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 09:10:21,639 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 09:10:21,639 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 09:10:21,642 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:21,642 AM [cluster-test-shard-dispatcher-15] [INFO] IsolatedLeader - member-1-shard-people-oper_testRemoveShardReplica: IsolatedLeader member-1-shard-people-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 09:10:21,643 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 09:10:21,643 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 09:10:21,643 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:21,643 AM [cluster-test-shard-dispatcher-16] [INFO] IsolatedLeader - member-1-shard-cars-oper_testRemoveShardReplica: IsolatedLeader member-1-shard-cars-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 09:10:21,643 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 09:10:21,643 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 09:10:21,643 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 09:10:21,643 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:21,643 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from IsolatedLeader to Leader 09:10:21,644 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 09:10:21,644 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from IsolatedLeader to Leader 09:10:21,644 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:10:21,647 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 326, lastApplied : -1, commitIndex : -1 09:10:21,648 AM [cluster-test-shard-dispatcher-16] [WARN] Leader - member-1-shard-cars-config_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 09:10:21,648 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:10:21,648 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:21,648 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 09:10:21,648 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 09:10:21,651 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 09:10:21,652 AM [cluster-test-shard-dispatcher-16] [INFO] IsolatedLeader - member-1-shard-cars-config_testRemoveShardReplica: IsolatedLeader member-1-shard-cars-config_testRemoveShardReplica switching from IsolatedLeader to Leader 09:10:21,652 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 09:10:21,652 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:21,652 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 09:10:21,652 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from IsolatedLeader to Leader 09:10:21,652 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 09:10:21,736 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:10:21,941 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:10:21,941 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:10:21,960 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 312, lastApplied : -1, commitIndex : -1 09:10:21,960 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 312, lastApplied : -1, commitIndex : -1 09:10:21,960 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 09:10:21,960 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 09:10:21,970 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 09:10:22,260 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:22,260 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:22,281 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:22,281 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:22,281 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:22,281 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:22,289 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 09:10:22,439 AM [cluster-test-pekko.actor.default-dispatcher-18] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Down)]. 09:10:22,439 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:22,439 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:22,453 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:22,453 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:22,581 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:22,582 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:22,601 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:22,601 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:22,601 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:22,601 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:22,609 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 09:10:22,666 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2559] 09:10:22,668 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:22,668 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:22,670 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-5233942713144746308] but current is [-8507291740446489021]. Reason: Cluster member removed, previous status [Down] 09:10:22,757 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:10:22,810 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:22,810 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:22,810 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-5233942713144746308] but current is [-8507291740446489021]. Reason: Cluster member removed, previous status [Down] 09:10:22,900 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:22,900 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:22,920 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:22,920 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:22,920 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:10:22,921 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:22,929 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 09:10:23,220 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:23,221 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:23,241 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:23,241 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:23,241 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:23,241 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:23,249 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 09:10:23,541 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:23,542 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:10:23,560 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:23,560 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:23,560 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:23,560 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:23,569 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 09:10:23,777 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:10:23,782 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:23,783 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:23,783 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:10:23,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:10:23,783 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:10:23,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:10:23,783 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:10:23,783 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:23,783 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:10:23,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 09:10:23,783 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:10:23,783 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 09:10:23,784 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 09:10:23,784 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 09:10:23,784 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 09:10:23,784 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 09:10:23,784 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 09:10:23,783 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:23,784 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 09:10:23,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:10:23,784 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:10:23,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:10:23,784 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 09:10:23,784 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:23,784 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:10:23,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 09:10:23,784 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:10:23,784 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} is now ready 09:10:23,784 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 09:10:23,784 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 09:10:23,785 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 09:10:23,785 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 09:10:23,785 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 09:10:23,785 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-632194329] 09:10:23,785 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:23,785 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1132474666] 09:10:23,785 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1132474666] 09:10:23,786 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-2}, datastoreType Config 09:10:23,786 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-2}] 09:10:23,787 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:23,787 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1057930693, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2cd5d79e] 09:10:23,788 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1057930693 for shard member-2-shard-cars-config_testRemoveShardReplica 09:10:23,789 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:23,790 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 09:10:23,790 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 09:10:23,790 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 09:10:23,790 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1563279735] 09:10:23,790 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:23,790 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1057930693] 09:10:23,790 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#1057930693] 09:10:23,791 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-632194329] 09:10:23,791 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:23,791 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1132474666] 09:10:23,792 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1132474666] with 27000 ms timeout 09:10:23,792 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2 : Local Shard replica for shard cars has been removed 09:10:23,792 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 09:10:23,792 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Stopping Shard 09:10:23,793 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal closed 09:10:23,793 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1132474666] 09:10:23,841 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-632194329] 09:10:23,841 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:23,841 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Local shard cars not found - shards present: [people] 09:10:23,842 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 09:10:23,842 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 09:10:23,842 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 09:10:23,843 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Stopping Shard 09:10:23,844 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal closed 09:10:23,844 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:10:23,844 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:10:23,845 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} 09:10:23,845 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 09:10:23,846 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 09:10:23,846 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:10:23,846 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Stopping Shard 09:10:23,846 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Stopping Shard 09:10:23,846 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal closed 09:10:23,846 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal closed 09:10:23,847 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:10:23,847 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:10:23,848 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:23,849 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:23,855 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:23,855 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:23,856 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:23,856 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:23,856 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:10:23,863 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:10:23,876 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:10:23,877 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardReplica 09:10:23,878 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 09:10:23,879 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Stopping Shard 09:10:23,879 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardReplica: journal closed 09:10:23,880 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:10:23,880 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:10:23,880 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 09:10:23,880 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:23,880 AM [cluster-test-shard-dispatcher-16] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:10:23,881 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardReplica 09:10:23,881 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardReplica 09:10:23,881 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:10:23,881 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Stopping Shard 09:10:23,881 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Stopping Shard 09:10:23,881 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardReplica: journal closed 09:10:23,882 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardReplica: journal closed 09:10:23,882 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:10:23,882 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:10:23,884 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:23,884 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:24,703 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:10:24,867 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-8507291740446489021] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:24,898 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [1300322287080634739] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:24,905 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:24,906 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:24,907 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:10:24.910] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:24,924 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 09:10:24,925 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveShardReplica 09:10:24,925 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardReplica 09:10:24,925 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:10:24,925 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@28ea46dc 09:10:24,925 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@28ea46dc 09:10:24,925 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Stopping Shard 09:10:24,925 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2d3422b4 09:10:24,925 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5d867cad 09:10:24,926 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5d867cad 09:10:24,926 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 09:10:24,926 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardReplica: journal closed 09:10:25,458 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: leader transfer aborted 09:10:25,458 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: Failed to transfer leadership in 532.9 ms 09:10:25,458 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In unpauseLeader 09:10:25,459 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Stopping Shard 09:10:25,460 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardReplica: journal closed 09:10:25,460 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:10:25,460 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:10:25,461 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 09:10:25,461 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardReplica 09:10:25,461 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardReplica 09:10:25,461 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:10:25,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58bcbc5f 09:10:25,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@58bcbc5f 09:10:25,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 09:10:25,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5c833390 09:10:25,462 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1470f2db 09:10:25,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5c833390 09:10:25,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 09:10:25,462 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@727b6e4d 09:10:25,798 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: leader transfer aborted 09:10:25,798 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: Failed to transfer leadership in 335.9 ms 09:10:25,798 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In unpauseLeader 09:10:25,798 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer aborted 09:10:25,798 AM [cluster-test-shard-dispatcher-14] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: Failed to transfer leadership in 337.0 ms 09:10:25,798 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In unpauseLeader 09:10:25,799 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Stopping Shard 09:10:25,799 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Stopping Shard 09:10:25,799 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardReplica: journal closed 09:10:25,799 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardReplica: journal closed 09:10:25,800 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:10:25,800 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:10:25,801 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:25,801 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:25,804 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:10:25.807] [cluster-test-pekko.actor.internal-dispatcher-8] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:25,840 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:10:25,852 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-196494348277637654] 09:10:25,852 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:10:25,855 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:10:25,855 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:10:25,858 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:25,858 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:25,873 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:10:25,873 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:10:25,873 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:10:25,875 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:25,877 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:25,879 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 09:10:25,883 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1684246098] 09:10:25,885 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1684246098] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:25,885 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 09:10:25,886 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:25,888 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:25,891 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 09:10:25,893 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:10:25,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:10:25,893 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:10:25,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:25,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveAllShardReplicas 09:10:25,894 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@50be0261} 09:10:25,894 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:25,894 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:25,894 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:10:25,894 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:25,894 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:25,900 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:25,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:25,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-1-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1011992482] 09:10:25,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveAllShardReplicas 09:10:25,901 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:10:25,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:10:25,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-1-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1093052729] 09:10:25,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1011992482] 09:10:25,902 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1011992482] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:25,902 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:25,902 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#543147162] 09:10:25,902 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1093052729] 09:10:25,902 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1093052729] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:25,902 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#543147162] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:25,904 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#-444071338 created and ready for shard:member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:25,904 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#-444071338] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:25,904 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:25,905 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:25,905 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:10:25,905 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:25,905 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#828474166 created and ready for shard:member-1-shard-people-config_testRemoveAllShardReplicas 09:10:25,906 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#828474166] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:25,905 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 09:10:25,906 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:10:25,906 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:10:25,906 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:25,907 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:25,909 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:25,911 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:25,911 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:25,911 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:25,911 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:25,911 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@261ad41d} 09:10:25,911 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:25,911 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:25,912 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:25,912 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:25,912 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-1-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1210833934] 09:10:25,912 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:25,912 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:25,912 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:25,912 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-1-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#-1557965769] 09:10:25,912 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1210833934] 09:10:25,913 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:25,913 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:25,913 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testRemoveAllShardReplicas: Recovery completed in in 3.401 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:25,913 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:25,914 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:25,915 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:25,916 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-people-config_testRemoveAllShardReplicas: Recovery completed in in 4.500 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:25,916 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:25,916 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:25,916 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 09:10:25,916 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#1210833934] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:25,916 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:10:25,916 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveAllShardReplicas is initialized 09:10:25,916 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#-1121213927 created and ready for shard:member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:25,917 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:10:25,917 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveAllShardReplicas is initialized 09:10:25,917 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#-1121213927] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:25,917 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#-1557965769] 09:10:25,917 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:10:25,917 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:25,917 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:25,918 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#-927500202 created and ready for shard:member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:25,919 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:25,919 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:25,919 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:25,919 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 09:10:25,919 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:10:25,921 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#-1557965769] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:25,921 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#-927500202] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:25,921 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:25,922 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed in in 6.974 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:25,922 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:25,922 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:25,922 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:10:25,922 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveAllShardReplicas is initialized 09:10:25,922 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 09:10:25,923 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 09:10:25,923 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from null to Follower 09:10:25,923 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from null to Follower 09:10:25,923 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 09:10:25,924 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-1-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:25,924 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-1-shard-people-oper_testRemoveAllShardReplicas: Recovery completed in in 5.493 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:25,925 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:25,925 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:25,925 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:10:25,925 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveAllShardReplicas is initialized 09:10:25,925 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 09:10:25,927 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:25,927 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 09:10:25,927 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from null to Follower 09:10:25,936 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-4830361462559740589] 09:10:25,936 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.4.0] ... 09:10:25,939 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:25,939 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:10:25,940 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:25,940 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:25,952 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:25,959 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:25,961 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 09:10:25,967 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 09:10:25,967 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:25,968 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1116475435] 09:10:25,984 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:10:25,984 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:25,984 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 09:10:25,984 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 09:10:25,984 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveAllShardReplicas 09:10:25,985 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveAllShardReplicas 09:10:25,985 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@1c0554c2} 09:10:25,985 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:25,985 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveAllShardReplicas 09:10:25,988 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 09:10:25,989 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1116475435] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:25,999 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveAllShardReplicas 09:10:26,0 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-1773844000] 09:10:26,0 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-1773844000] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,0 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:26,1 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-2-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:26,4 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:26,5 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#-2016313153 created and ready for shard:member-2-shard-cars-config_testRemoveAllShardReplicas 09:10:26,5 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#-2016313153] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-378085898] 09:10:26,5 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-378085898] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,6 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:26,7 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-2-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:26,8 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-2-shard-cars-config_testRemoveAllShardReplicas: Recovery completed in in 3.595 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:26,8 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#395201162 created and ready for shard:member-2-shard-people-config_testRemoveAllShardReplicas 09:10:26,9 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#395201162] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,9 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-2-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:26,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1082137410] 09:10:26,11 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1082137410] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,14 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 09:10:26,18 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:26,19 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:10:26,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 09:10:26,20 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 09:10:26,20 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,20 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveAllShardReplicas 09:10:26,20 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@71ab69f9} 09:10:26,21 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-2-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:26,22 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-2-shard-people-config_testRemoveAllShardReplicas: Recovery completed in in 3.085 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:26,22 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:26,22 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,22 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:26,22 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:26,22 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 09:10:26,22 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:10:26,23 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveAllShardReplicas is initialized 09:10:26,23 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:10:26,23 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveAllShardReplicas is initialized 09:10:26,25 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:26,25 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:26,25 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas#2046265123] 09:10:26,25 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:10:26,25 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 09:10:26,25 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:10:26,25 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:26,26 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:26,26 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#1585322750 created and ready for shard:member-2-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,28 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas#2046265123] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveAllShardReplicas 09:10:26,30 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas#595630455] 09:10:26,30 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:26,30 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-2-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:26,31 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:10:26,31 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#-1932375163 created and ready for shard:member-2-shard-people-oper_testRemoveAllShardReplicas 09:10:26,31 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#1585322750] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,31 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas#595630455] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,31 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#-1932375163] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,31 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:26,34 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:26,39 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:26,40 AM [cluster-test-shard-dispatcher-22] [INFO] Recovery - member-2-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:26,41 AM [cluster-test-shard-dispatcher-22] [INFO] Recovery - member-2-shard-people-oper_testRemoveAllShardReplicas: Recovery completed in in 6.792 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:26,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:26,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:26,43 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:26,43 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,43 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,43 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:26,43 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:10:26,43 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:10:26,43 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveAllShardReplicas is initialized 09:10:26,43 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:26,43 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-2-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:26,43 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 09:10:26,43 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 09:10:26,43 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveAllShardReplicas from null to Follower 09:10:26,44 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-2-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed in in 12.17 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:26,45 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:10:26,45 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveAllShardReplicas is initialized 09:10:26,46 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:26,46 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:26,46 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 09:10:26,46 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,46 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 09:10:26,46 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 09:10:26,46 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveAllShardReplicas from null to Follower 09:10:26,46 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveAllShardReplicas from null to Follower 09:10:26,46 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:26,46 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:26,46 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:10:26,46 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:10:26,46 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:10:26,46 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-2-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:10:26,46 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:26,47 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:26,64 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-3302345017980981098] 09:10:26,65 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.4.0] ... 09:10:26,68 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:26,68 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:10:26,76 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:26,76 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:26,84 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:26,86 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:26,89 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 09:10:26,96 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#994970029] 09:10:26,97 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:10:26,97 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 09:10:26,97 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 09:10:26,97 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveAllShardReplicas 09:10:26,97 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveAllShardReplicas 09:10:26,98 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@23abd8ad} 09:10:26,98 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 09:10:26,98 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#994970029] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,98 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:26,100 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:26,103 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 09:10:26,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:26,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveAllShardReplicas 09:10:26,106 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveAllShardReplicas 09:10:26,106 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1440728504] 09:10:26,107 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:26,107 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1440728504] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,107 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 09:10:26,107 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#-849724299 created and ready for shard:member-3-shard-cars-config_testRemoveAllShardReplicas 09:10:26,108 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 09:10:26,108 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:26,108 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#1556115840] 09:10:26,108 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#-849724299] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,109 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#1556115840] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,109 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:26,111 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-3-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:26,112 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-3-shard-cars-config_testRemoveAllShardReplicas: Recovery completed in in 2.300 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:26,112 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1299833406] 09:10:26,113 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1299833406] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,113 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:26,114 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:10:26,114 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 09:10:26,114 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 09:10:26,114 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,114 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveAllShardReplicas 09:10:26,115 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@7570e6fc} 09:10:26,115 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:26,115 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,116 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:26,116 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:26,116 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:10:26,116 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveAllShardReplicas is initialized 09:10:26,116 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 09:10:26,116 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#1535241468] 09:10:26,116 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:10:26,116 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:26,117 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#1535241468] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,117 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:26,117 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#1932172184 created and ready for shard:member-3-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,117 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#1932172184] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,119 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:26,120 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveAllShardReplicas from null to Follower 09:10:26,120 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#1944972299 created and ready for shard:member-3-shard-people-config_testRemoveAllShardReplicas 09:10:26,121 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#1944972299] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,122 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-3-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:26,122 AM [cluster-test-shard-dispatcher-26] [INFO] Recovery - member-3-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:26,123 AM [cluster-test-shard-dispatcher-26] [INFO] Recovery - member-3-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed in in 3.629 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:26,123 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:26,123 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:26,123 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 09:10:26,124 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:10:26,124 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:26,124 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveAllShardReplicas 09:10:26,125 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:10:26,125 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveAllShardReplicas is initialized 09:10:26,125 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-1503614189] 09:10:26,126 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:26,126 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-1503614189] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,126 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-3-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:26,127 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:26,128 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:26,129 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:26,129 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:26,129 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,129 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,129 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 09:10:26,129 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#-50565898 created and ready for shard:member-3-shard-people-oper_testRemoveAllShardReplicas 09:10:26,129 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:26,129 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:26,129 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:10:26,129 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#-50565898] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:26,129 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:26,129 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,130 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:26,130 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:10:26,130 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 09:10:26,130 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:10:26,130 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:10:26,130 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-3-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:26,131 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-3-shard-people-oper_testRemoveAllShardReplicas: Recovery completed in in 2.574 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:26,131 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:26,131 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:26,131 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 09:10:26,131 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:26,130 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-people-config_testRemoveAllShardReplicas: Recovery completed in in 5.694 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:26,131 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:10:26,131 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is initialized 09:10:26,131 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 09:10:26,131 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:10:26,131 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveAllShardReplicas is initialized 09:10:26,131 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveAllShardReplicas from null to Follower 09:10:26,131 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:26,131 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:26,131 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 09:10:26,131 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:10:26,131 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:10:26,131 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveAllShardReplicas from null to Follower 09:10:26,259 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 09:10:26,259 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:26,259 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:26,259 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:26,259 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 09:10:26,259 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Follower to Candidate 09:10:26,309 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 09:10:26,309 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:26,309 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:26,309 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:26,309 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 09:10:26,309 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Follower to Candidate 09:10:26,319 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 09:10:26,319 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:26,319 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:26,319 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:26,319 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 09:10:26,319 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Follower to Candidate 09:10:26,339 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 09:10:26,340 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:26,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:26,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:26,340 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 09:10:26,340 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Follower to Candidate 09:10:26,648 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 09:10:26,668 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 09:10:26,728 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 09:10:26,749 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 09:10:26,877 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:10:26,877 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:10:26,878 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:26,878 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:26,879 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:26,879 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:10:26,879 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:10:26,880 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:26,880 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:10:26,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:26,880 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:10:26,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 09:10:26,880 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:10:26,880 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:10:26,880 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-1-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:10:26,880 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-1-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:26,888 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:10:26,888 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:10:26,977 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:26,977 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:26,977 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:10:26,977 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:10:26,977 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,977 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:10:26,977 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,977 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:10:26,977 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:10:26,977 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:26,978 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:10:26,978 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:10:26,978 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 09:10:26,978 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:26,978 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:10:26,978 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,978 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 09:10:26,978 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,978 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:10:26,978 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 09:10:26,978 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 09:10:26,978 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 09:10:26,978 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 09:10:26,978 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 09:10:27,39 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 09:10:27,41 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:27,42 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:27,47 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:27,47 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 09:10:27,47 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 09:10:27,47 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 09:10:27,47 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@11ee3aae 09:10:27,48 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Candidate to Leader 09:10:27,49 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 09:10:27,49 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@212ca6d4 09:10:27,49 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done false 09:10:27,50 AM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:27,50 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2ee77653 09:10:27,51 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveAllShardReplicas status sync done false 09:10:27,51 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:27,57 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:27,57 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 09:10:27,57 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: became leader with frontend state for [] 09:10:27,57 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 09:10:27,57 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1b3bc437 09:10:27,58 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Candidate to Leader 09:10:27,59 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done false 09:10:27,59 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7b26f75e 09:10:27,59 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c5770f5 09:10:27,59 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveAllShardReplicas status sync done false 09:10:27,89 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:10:27,141 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 09:10:27,143 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:27,143 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 09:10:27,144 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:27,145 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:27,146 AM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:27,156 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:27,156 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 09:10:27,156 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: became leader with frontend state for [] 09:10:27,156 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 09:10:27,156 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@15fc2920 09:10:27,156 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Candidate to Leader 09:10:27,156 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 09:10:27,156 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} is now ready 09:10:27,156 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1684246098] 09:10:27,157 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:27,157 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1011992482] 09:10:27,157 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2a6202aa 09:10:27,158 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 09:10:27,158 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveAllShardReplicas status sync done false 09:10:27,158 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1011992482] 09:10:27,158 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done false 09:10:27,158 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7654c4eb 09:10:27,158 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1116475435] 09:10:27,159 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 09:10:27,159 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:27,159 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-1773844000] 09:10:27,159 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-1773844000] 09:10:27,159 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#994970029] 09:10:27,159 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:27,159 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1440728504] 09:10:27,160 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1440728504] 09:10:27,160 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:27,160 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 09:10:27,161 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 09:10:27,161 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 09:10:27,161 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@11da9a21 09:10:27,161 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Candidate to Leader 09:10:27,161 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 09:10:27,161 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@24a5f996 09:10:27,162 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 09:10:27,162 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveAllShardReplicas status sync done false 09:10:27,162 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done false 09:10:27,162 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1a7eb7b1 09:10:27,162 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 09:10:27,171 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@43c4e16c] 09:10:27,172 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 09:10:27,174 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#2064687210] 09:10:27,174 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:27,175 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas-notifier#-289567483 created and ready for shard:member-1-shard-pets-config_testRemoveAllShardReplicas 09:10:27,176 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-1-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:27,177 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:27,180 AM [cluster-test-shard-dispatcher-20] [INFO] Recovery - member-1-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:27,181 AM [cluster-test-shard-dispatcher-20] [INFO] Recovery - member-1-shard-pets-config_testRemoveAllShardReplicas: Recovery completed in in 3.100 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:27,181 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:27,181 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:27,181 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 09:10:27,181 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 09:10:27,181 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testRemoveAllShardReplicas is initialized 09:10:27,181 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:10:27,182 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from null to Follower 09:10:27,185 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@46c7a519] 09:10:27,186 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 09:10:27,187 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-429831355] 09:10:27,187 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:27,188 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-2-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:27,188 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas-notifier#-1760883083 created and ready for shard:member-2-shard-pets-config_testRemoveAllShardReplicas 09:10:27,191 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:27,193 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-2-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:27,194 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-2-shard-pets-config_testRemoveAllShardReplicas: Recovery completed in in 2.655 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:27,194 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 09:10:27,194 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testRemoveAllShardReplicas is initialized 09:10:27,194 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:27,194 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:27,194 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 09:10:27,194 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:10:27,194 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testRemoveAllShardReplicas from null to Follower 09:10:27,198 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@209a69da] 09:10:27,199 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3 doCreateShard: shardId: member-3-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 09:10:27,200 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-150345577] 09:10:27,200 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1684246098] 09:10:27,200 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 09:10:27,201 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#2064687210] 09:10:27,201 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 09:10:27,201 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#2064687210] 09:10:27,201 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1116475435] 09:10:27,201 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 09:10:27,201 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-429831355] 09:10:27,201 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-3-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 09:10:27,202 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas-notifier#-2057928164 created and ready for shard:member-3-shard-pets-config_testRemoveAllShardReplicas 09:10:27,202 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-429831355] 09:10:27,202 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#994970029] 09:10:27,202 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 09:10:27,203 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 09:10:27,205 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-3-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 09:10:27,206 AM [cluster-test-shard-dispatcher-21] [INFO] Recovery - member-3-shard-pets-config_testRemoveAllShardReplicas: Recovery completed in in 2.613 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:27,206 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:27,206 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:27,206 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 09:10:27,203 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 09:10:27,208 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [pets] 09:10:27,208 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized 09:10:27,208 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized - notifying 1 OnShardInitialized callbacks 09:10:27,208 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-150345577] 09:10:27,208 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-150345577] 09:10:27,209 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 09:10:27,209 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-pets-config_testRemoveAllShardReplicas from null to Follower 09:10:27,211 AM [main] [INFO] ClusterAdminRpcService - Removing replicas for all shards 09:10:27,211 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testRemoveAllShardReplicas 09:10:27,211 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testRemoveAllShardReplicas 09:10:27,212 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 09:10:27,212 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 09:10:27,212 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 09:10:27,212 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 09:10:27,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:27,213 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=pets, memberName=MemberName{name=member-3}] 09:10:27,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, primaryVersion=13] 09:10:27,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:10:27,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, primaryVersion=13] 09:10:27,213 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:27,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas for shard member-3-shard-cars-oper_testRemoveAllShardReplicas 09:10:27,214 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, primaryVersion=13] 09:10:27,214 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:10:27,214 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, primaryVersion=13] 09:10:27,214 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 09:10:27,215 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas for shard member-3-shard-people-oper_testRemoveAllShardReplicas 09:10:27,215 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 09:10:27,215 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas for shard member-3-shard-cars-config_testRemoveAllShardReplicas 09:10:27,216 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas for shard member-3-shard-people-config_testRemoveAllShardReplicas 09:10:27,218 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testRemoveAllShardReplicas} for shard cars 09:10:27,218 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard cars 09:10:27,218 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 09:10:27,219 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 1 09:10:27,219 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-oper_testRemoveAllShardReplicas} for shard people 09:10:27,219 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard people 09:10:27,219 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 09:10:27,219 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 2 09:10:27,220 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveAllShardReplicas} for shard cars 09:10:27,220 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 09:10:27,220 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 09:10:27,220 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 3 09:10:27,222 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:27,223 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#1535241468] with 27000 ms timeout 09:10:27,223 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard cars has been removed 09:10:27,223 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [people] 09:10:27,224 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-config_testRemoveAllShardReplicas} for shard people 09:10:27,224 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard people 09:10:27,224 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 09:10:27,224 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 09:10:27,224 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveAllShardReplicas: journal closed 09:10:27,224 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 4 09:10:27,225 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done true 09:10:27,225 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:27,225 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#1535241468] 09:10:27,225 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:27,226 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1440728504] with 27000 ms timeout 09:10:27,226 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 09:10:27,226 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets, people] 09:10:27,227 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 09:10:27,227 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done true 09:10:27,227 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done true 09:10:27,227 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveAllShardReplicas: journal closed 09:10:27,227 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:27,228 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#-1440728504] 09:10:27,228 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done true 09:10:27,228 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-1503614189] with 27000 ms timeout 09:10:27,228 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard people has been removed 09:10:27,228 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [] 09:10:27,228 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#1556115840] with 27000 ms timeout 09:10:27,228 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 09:10:27,229 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard people has been removed 09:10:27,229 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets] 09:10:27,229 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 09:10:27,229 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveAllShardReplicas: journal closed 09:10:27,229 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveAllShardReplicas: journal closed 09:10:27,229 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-1503614189] 09:10:27,229 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#1556115840] 09:10:27,598 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 09:10:27,598 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:27,598 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 09:10:27,598 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:27,599 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 09:10:27,599 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Follower to Candidate 09:10:27,600 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:27,600 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:27,611 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:10:27,611 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 09:10:27,611 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: became leader with frontend state for [] 09:10:27,611 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 09:10:27,611 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@52a84548 09:10:27,612 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Candidate to Leader 09:10:27,612 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 09:10:27,613 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@89b45ea 09:10:27,613 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 09:10:27,613 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, primaryVersion=13] 09:10:27,613 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 09:10:27,613 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-pets-config_testRemoveAllShardReplicas status sync done false 09:10:27,614 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@41cdc6dd 09:10:27,614 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 09:10:27,614 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done false 09:10:27,614 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas for shard member-3-shard-pets-config_testRemoveAllShardReplicas 09:10:27,619 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-pets-config_testRemoveAllShardReplicas} for shard pets 09:10:27,619 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard pets 09:10:27,620 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=pets}, type Config 09:10:27,620 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 5 09:10:27,622 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1684246098] 09:10:27,623 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:27,623 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1011992482] 09:10:27,623 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1011992482] 09:10:27,623 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1684246098] 09:10:27,624 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:10:27,624 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1093052729] 09:10:27,624 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#1093052729] 09:10:27,624 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorVotingConfigSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:27,624 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1684246098] 09:10:27,624 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 09:10:27,624 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#2064687210] 09:10:27,624 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#2064687210] 09:10:27,625 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1116475435] 09:10:27,625 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:27,625 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-1773844000] 09:10:27,625 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#-1773844000] 09:10:27,625 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1116475435] 09:10:27,625 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:10:27,625 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-378085898] 09:10:27,625 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-378085898] 09:10:27,625 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done true 09:10:27,626 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1116475435] 09:10:27,626 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-150345577] with 27000 ms timeout 09:10:27,626 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard pets has been removed 09:10:27,626 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [] 09:10:27,626 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 09:10:27,626 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 09:10:27,626 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-429831355] 09:10:27,627 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-429831355] 09:10:27,627 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#994970029] 09:10:27,627 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-3-shard-pets-config_testRemoveAllShardReplicas: journal closed 09:10:27,627 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#-150345577] 09:10:27,628 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:27,628 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [] 09:10:27,628 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#994970029] 09:10:27,628 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:10:27,628 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard people not found - shards present: [] 09:10:27,628 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#994970029] 09:10:27,628 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 09:10:27,628 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard pets not found - shards present: [] 09:10:27,629 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 09:10:27,629 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 0 shards 09:10:27,629 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:10:27,629 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:10:27,629 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 09:10:27,630 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 0 shards 09:10:27,630 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:10:27,630 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:10:27,631 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:27,631 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:27,637 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:27,638 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:27,638 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:27,638 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:27,642 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:10:27,649 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 09:10:27,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testRemoveAllShardReplicas 09:10:27,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveAllShardReplicas 09:10:27,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveAllShardReplicas 09:10:27,650 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 09:10:27,651 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 09:10:27,651 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 09:10:27,651 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 09:10:27,651 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveAllShardReplicas: journal closed 09:10:27,651 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testRemoveAllShardReplicas: journal closed 09:10:27,652 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveAllShardReplicas: journal closed 09:10:27,652 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:10:27,653 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:10:27,653 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 09:10:27,653 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveAllShardReplicas 09:10:27,653 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveAllShardReplicas 09:10:27,653 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:10:27,653 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 09:10:27,654 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 09:10:27,654 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveAllShardReplicas: journal closed 09:10:27,654 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveAllShardReplicas: journal closed 09:10:27,655 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:10:27,655 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:10:27,655 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:27,655 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:27,896 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:10:28,646 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-3302345017980981098] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:28,676 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-4830361462559740589] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:28,678 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:28,679 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:28,681 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:10:28.684] [cluster-test-pekko.actor.internal-dispatcher-8] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:28,703 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 09:10:28,703 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testRemoveAllShardReplicas 09:10:28,703 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveAllShardReplicas 09:10:28,703 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveAllShardReplicas 09:10:28,703 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 09:10:28,704 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6bb92370 09:10:28,704 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2bf788a2 09:10:28,704 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6bb92370 09:10:28,704 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2bf788a2 09:10:28,704 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 09:10:28,704 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 09:10:28,704 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@253073ac 09:10:28,704 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5bab0a1d 09:10:28,704 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@210fe3d2 09:10:28,704 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-pets-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5bab0a1d 09:10:28,704 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@640d7997 09:10:28,704 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 09:10:29,56 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer aborted 09:10:29,56 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: Failed to transfer leadership in 352.3 ms 09:10:29,56 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In unpauseLeader 09:10:29,57 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 09:10:29,57 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveAllShardReplicas: journal closed 09:10:29,206 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer aborted 09:10:29,206 AM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: Failed to transfer leadership in 502.6 ms 09:10:29,206 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In unpauseLeader 09:10:29,207 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 09:10:29,208 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testRemoveAllShardReplicas: journal closed 09:10:29,295 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer aborted 09:10:29,295 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: Failed to transfer leadership in 591.7 ms 09:10:29,295 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In unpauseLeader 09:10:29,296 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 09:10:29,297 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveAllShardReplicas: journal closed 09:10:29,297 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:10:29,297 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:10:29,298 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 09:10:29,299 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveAllShardReplicas 09:10:29,299 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveAllShardReplicas 09:10:29,299 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:10:29,299 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4bb98625 09:10:29,299 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@743cbfd8 09:10:29,299 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@743cbfd8 09:10:29,299 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 09:10:29,299 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1f0e3e48 09:10:29,299 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@73284f2c 09:10:29,299 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@73284f2c 09:10:29,300 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 09:10:29,606 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer aborted 09:10:29,606 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 306.9 ms 09:10:29,606 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In unpauseLeader 09:10:29,607 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 09:10:29,608 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveAllShardReplicas: journal closed 09:10:29,716 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer aborted 09:10:29,716 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 416.3 ms 09:10:29,716 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In unpauseLeader 09:10:29,716 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 09:10:29,717 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveAllShardReplicas: journal closed 09:10:29,717 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:10:29,717 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:10:29,719 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:29,719 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:29,721 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:10:29.724] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:29,753 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:29,763 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [7293231309246469332] 09:10:29,763 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:10:29,766 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:10:29,766 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:10:29,768 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:29,768 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:29,780 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:29,783 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:29,785 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:10:29,785 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:10:29,785 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:10:29,785 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplicaFailures 09:10:29,787 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-515184166] 09:10:29,788 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:10:29,788 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:10:29,788 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:10:29,788 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplicaFailures 09:10:29,789 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:29,789 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 09:10:29,791 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-515184166] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,794 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@53794475} 09:10:29,794 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} started 09:10:29,794 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:29,794 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:29,795 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplicaFailures 09:10:29,796 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:29,800 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplicaFailures 09:10:29,803 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 09:10:29,803 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 09:10:29,803 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures on shard member-1-shard-cars-config_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#428791207] 09:10:29,803 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#428791207] 09:10:29,804 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Shard created, persistent : true 09:10:29,804 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#-827647648] 09:10:29,804 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:10:29,805 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:10:29,805 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:10:29,805 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplicaFailures 09:10:29,805 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#1432540961 created and ready for shard:member-1-shard-cars-config_testAddShardReplicaFailures 09:10:29,805 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#428791207] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,806 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:29,806 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 09:10:29,806 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testAddShardReplicaFailures: Starting recovery with journal batch size 1 09:10:29,807 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#-827647648] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,807 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#1432540961] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,808 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} started 09:10:29,809 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@7b57f0f} 09:10:29,809 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:29,809 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplicaFailures 09:10:29,810 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 09:10:29,810 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 09:10:29,810 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures on shard member-1-shard-cars-oper_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-2024959382] 09:10:29,810 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-2024959382] 09:10:29,810 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Shard created, persistent : true 09:10:29,811 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-2024959382] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,811 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#-1782433547 created and ready for shard:member-1-shard-cars-oper_testAddShardReplicaFailures 09:10:29,811 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#-1782433547] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,811 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-cars-oper_testAddShardReplicaFailures: Starting recovery with journal batch size 1 09:10:29,812 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplicaFailures: journal open: applyTo=0 09:10:29,812 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=people} 09:10:29,812 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: onAddShardReplica: AddShardReplica[ShardName=people] 09:10:29,812 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: No module configuration exists for shard people 09:10:29,813 AM [cluster-test-pekko.actor.default-dispatcher-12] [ERROR] ClusterAdminRpcService - Failed to add replica for shard ShardName{value=people} java.lang.IllegalArgumentException: No module configuration exists for shard people at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onAddShardReplica(ShardManager.java:1125) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleReceive(ShardManager.java:257) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:37) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:37) at scala.PartialFunction.applyOrElse(PartialFunction.scala:225) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:224) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:33) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:280) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:281) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:481) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:273) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:234) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:246) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:10:29,813 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplicaFailures: journal open: applyTo=0 09:10:29,813 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 09:10:29,814 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplicaFailures 09:10:29,814 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 09:10:29,815 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testAddShardReplicaFailures: Local TermInfo store seeded with TermInfo{term=0} 09:10:29,815 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testAddShardReplicaFailures: Recovery completed in in 3.072 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:29,815 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:10:29,816 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplicaFailures is initialized 09:10:29,816 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 09:10:29,816 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:29,816 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from null to Follower 09:10:29,816 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-cars-oper_testAddShardReplicaFailures: Local TermInfo store seeded with TermInfo{term=0} 09:10:29,817 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-cars-oper_testAddShardReplicaFailures: Recovery completed in in 3.418 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:29,817 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:10:29,817 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplicaFailures is initialized 09:10:29,817 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 09:10:29,817 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:29,817 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from null to Follower 09:10:29,819 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Starting new election term 1 09:10:29,819 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:29,819 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 09:10:29,819 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:29,820 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Follower to Candidate 09:10:29,820 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:10:29,820 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 09:10:29,820 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Starting new election term 1 09:10:29,820 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:29,820 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 09:10:29,820 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:29,820 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Follower to Candidate 09:10:29,820 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:10:29,820 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 09:10:29,820 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: became leader with frontend state for [] 09:10:29,820 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Candidate to Leader 09:10:29,820 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures 09:10:29,820 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@72f013ae 09:10:29,821 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplicaFailures from Candidate to Leader 09:10:29,821 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplicaFailures is ready 09:10:29,821 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: became leader with frontend state for [] 09:10:29,821 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Candidate to Leader 09:10:29,821 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures 09:10:29,821 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c17bef9 09:10:29,821 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Stopping Shard 09:10:29,821 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplicaFailures from Candidate to Leader 09:10:29,821 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplicaFailures is ready 09:10:29,821 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplicaFailures: journal closed 09:10:29,821 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:10:29,822 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:10:29,822 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-515184166] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-515184166] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,822 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 09:10:29,822 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f52cc8403d0] to Actor[pekko://cluster-test/user/$a#488636780] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#488636780] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,822 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-515184166] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#-515184166] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,822 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplicaFailures 09:10:29,823 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 09:10:29,823 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@7c37f380 09:10:29,823 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@7c37f380 09:10:29,823 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Stopping Shard 09:10:29,823 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplicaFailures: journal closed 09:10:29,823 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:10:29,824 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:10:29,824 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#-827647648] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#-827647648] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,825 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:29,825 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:29,827 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:10:29,847 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:10:29,858 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-2182822994634547345] 09:10:29,859 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:10:29,861 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:10:29,861 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:10:29,862 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:29,862 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:29,872 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:10:29,872 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:10:29,873 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:10:29,873 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:29,875 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:29,877 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 09:10:29,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1068642326] 09:10:29,880 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:10:29,880 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:10:29,880 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:10:29,880 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplica 09:10:29,880 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:29,880 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:10:29,882 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@648c5df5} 09:10:29,882 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} started 09:10:29,882 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1068642326] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,882 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:29,882 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplica 09:10:29,885 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:10:29,885 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:10:29,885 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica on shard member-1-shard-cars-config_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596] 09:10:29,885 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596] 09:10:29,885 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Shard created, persistent : true 09:10:29,885 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,886 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 09:10:29,886 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#-967500048 created and ready for shard:member-1-shard-cars-config_testAddShardReplica 09:10:29,886 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#-967500048] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,887 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplica: journal open: applyTo=0 09:10:29,889 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:29,890 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-config_testAddShardReplica: Recovery completed in in 2.526 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:29,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:29,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:29,890 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:10:29,890 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from null to Follower 09:10:29,890 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplica is initialized 09:10:29,892 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-config_testAddShardReplica (Candidate): Starting new election term 1 09:10:29,892 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:29,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:29,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:29,892 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Follower to Candidate 09:10:29,892 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:10:29,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:29,892 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: became leader with frontend state for [] 09:10:29,892 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Candidate to Leader 09:10:29,892 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 09:10:29,892 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7a11f686 09:10:29,893 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Candidate to Leader 09:10:29,893 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 09:10:29,932 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1068642326] 09:10:29,933 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:29,933 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596] 09:10:29,933 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:29,936 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:29,940 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 09:10:29,943 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1312307504] 09:10:29,944 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1312307504] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,944 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} started 09:10:29,945 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:10:29,945 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:10:29,945 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:10:29,946 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplica 09:10:29,946 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@5659c4e2} 09:10:29,946 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:29,946 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplica 09:10:29,948 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:29,948 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 09:10:29,948 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 09:10:29,948 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:29,948 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 09:10:29,948 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,949 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#-1504027263 created and ready for shard:member-1-shard-cars-oper_testAddShardReplica 09:10:29,949 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 09:10:29,949 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#-1504027263] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:29,950 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 09:10:29,953 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:29,953 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-oper_testAddShardReplica: Recovery completed in in 2.639 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:29,954 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:10:29,954 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplica is initialized 09:10:29,954 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:29,954 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:29,954 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from null to Follower 09:10:29,956 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testAddShardReplica (Candidate): Starting new election term 1 09:10:29,956 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:29,956 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:29,956 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:29,956 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:10:29,956 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:29,956 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: became leader with frontend state for [] 09:10:29,956 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Follower to Candidate 09:10:29,956 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Candidate to Leader 09:10:29,956 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 09:10:29,956 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@56a12ff2 09:10:29,957 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Candidate to Leader 09:10:29,957 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:29,994 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1312307504] 09:10:29,995 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:10:29,995 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:30,3 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:30,12 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-3332200726130069411] 09:10:30,12 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.4.0] ... 09:10:30,14 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:30,14 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:10:30,16 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:30,30 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:30,34 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:30,42 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:30,45 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 09:10:30,53 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#989578082] 09:10:30,54 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} started 09:10:30,54 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#989578082] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:30,54 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:10:30,54 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:30,54 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 09:10:30,54 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 09:10:30,55 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@62a08140} 09:10:30,55 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:30,56 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:30,57 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 09:10:30,58 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1009562672] 09:10:30,59 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:10:30,59 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 09:10:30,59 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 09:10:30,60 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1009562672] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:30,62 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@6c677364} 09:10:30,62 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} started 09:10:30,63 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:30,90 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:10:30,96 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:30,99 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:30,99 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 09:10:30,99 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:30,100 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:30,883 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:10:30,883 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:30,883 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 09:10:30,883 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:10:30,883 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 09:10:30,884 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:30,884 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 09:10:30,884 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 09:10:30,884 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:30,965 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 09:10:30,965 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 09:10:30,967 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:30,967 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica] 09:10:30,969 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:30,969 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596, primaryVersion=13] 09:10:30,973 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596 for shard member-2-shard-cars-config_testAddShardReplica 09:10:30,973 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#964720261] 09:10:30,973 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Shard created, persistent : true 09:10:30,973 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#964720261] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:30,974 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-2137248108 created and ready for shard:member-2-shard-cars-config_testAddShardReplica 09:10:30,974 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#-2137248108] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:30,974 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-2-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 09:10:30,975 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddShardReplica 09:10:30,975 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 09:10:30,975 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:10:30,976 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddShardReplica: journal open: applyTo=0 09:10:30,978 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: snapshot is durable as of 2026-03-18T09:10:30.975784726Z 09:10:30,979 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-2-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:30,979 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-2-shard-cars-config_testAddShardReplica: Recovery completed in in 3.210 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:30,979 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:10:30,980 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddShardReplica is initialized 09:10:30,980 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:30,980 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:30,980 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , received role change from null to Follower 09:10:30,980 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-2-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 09:10:30,981 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-2-shard-cars-config_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddShardReplica 09:10:30,982 AM [cluster-test-shard-dispatcher-15] [INFO] SnapshotManager - member-2-shard-cars-config_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:10:30,982 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 09:10:30,982 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 09:10:30,982 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@64cee6ba 09:10:30,982 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddShardReplica from null to Follower 09:10:30,982 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 09:10:30,984 AM [cluster-test-shard-dispatcher-15] [INFO] SnapshotManager - member-2-shard-cars-config_testAddShardReplica: snapshot is durable as of 2026-03-18T09:10:30.982499031Z 09:10:30,984 AM [cluster-test-shard-dispatcher-15] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddShardReplica: Applying snapshot 09:10:30,984 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:10:30,985 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#2008516498] with name member-2-shard-cars-config_testAddShardReplica-DTCL-publisher 09:10:30,985 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#2008516498] 09:10:30,985 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: state snapshot applied in 487.1 μs 09:10:30,985 AM [cluster-test-shard-dispatcher-15] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddShardReplica: Done applying snapshot 09:10:30,985 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 09:10:30,985 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#2008516498] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:30,985 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 99.50 μs 09:10:30,986 AM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:10:30,988 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6a090453 09:10:30,988 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 09:10:30,988 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596 09:10:30,988 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 09:10:30,989 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddShardReplica 09:10:30,989 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 09:10:30,991 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:30,992 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 09:10:30,992 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 09:10:30,992 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 09:10:30,992 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:30,992 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#989578082] 09:10:30,992 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:30,992 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#964720261] 09:10:30,992 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#964720261] 09:10:30,993 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1009562672] 09:10:30,993 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:10:30,993 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 09:10:30,993 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 09:10:30,993 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 09:10:30,993 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:30,993 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica] 09:10:30,995 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:30,995 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173, primaryVersion=13] 09:10:30,998 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173 for shard member-2-shard-cars-oper_testAddShardReplica 09:10:30,998 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#711783292] 09:10:30,998 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 09:10:30,998 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#711783292] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:30,999 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-2-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 09:10:30,999 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#233784321 created and ready for shard:member-2-shard-cars-oper_testAddShardReplica 09:10:30,999 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#233784321] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:31,0 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddShardReplica 09:10:31,0 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 09:10:31,1 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 09:10:31,1 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:10:31,3 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-2-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:31,3 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-2-shard-cars-oper_testAddShardReplica: Recovery completed in in 2.561 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:31,4 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:10:31,4 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddShardReplica is initialized 09:10:31,4 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:31,4 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:31,4 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , received role change from null to Follower 09:10:31,4 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-2-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 09:10:31,4 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2026-03-18T09:10:31.001554192Z 09:10:31,7 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-2-shard-cars-oper_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddShardReplica 09:10:31,7 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 09:10:31,7 AM [cluster-test-shard-dispatcher-23] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:10:31,7 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 09:10:31,8 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@287b4f31 09:10:31,8 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddShardReplica from null to Follower 09:10:31,8 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:31,10 AM [cluster-test-shard-dispatcher-23] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2026-03-18T09:10:31.008290758Z 09:10:31,10 AM [cluster-test-shard-dispatcher-23] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot 09:10:31,10 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:10:31,11 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#-926075714] with name member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher 09:10:31,11 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#-926075714] 09:10:31,11 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: state snapshot applied in 714.2 μs 09:10:31,11 AM [cluster-test-shard-dispatcher-23] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddShardReplica: Done applying snapshot 09:10:31,11 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 09:10:31,11 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 74.94 μs 09:10:31,11 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#-926075714] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:31,12 AM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 09:10:31,14 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173 09:10:31,14 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 09:10:31,15 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddShardReplica 09:10:31,15 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 09:10:31,17 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:31,18 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5d843b3e 09:10:31,18 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 09:10:31,18 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:31,18 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 09:10:31,18 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1009562672] 09:10:31,18 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 09:10:31,18 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:31,18 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:10:31,18 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#711783292] 09:10:31,18 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#711783292] 09:10:31,26 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:10:31,36 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-3966851390747401946] 09:10:31,37 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:31,37 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:31,37 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 09:10:31,37 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 09:10:31,37 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 09:10:31,37 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 09:10:31,37 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:31,37 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 09:10:31,38 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.4.0] ... 09:10:31,41 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:31,41 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:10:31,42 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:31,42 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:31,55 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:31,57 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:31,59 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 09:10:31,62 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1641538917] 09:10:31,63 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} started 09:10:31,64 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:31,65 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:31,67 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 09:10:31,70 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1641538917] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:31,71 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:10:31,71 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 09:10:31,71 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 09:10:31,71 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@1d894870} 09:10:31,71 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:31,71 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-576155259] 09:10:31,72 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:10:31,72 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 09:10:31,72 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 09:10:31,72 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-576155259] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:31,74 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@335dba03} 09:10:31,74 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} started 09:10:31,74 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:31,94 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:10:31,96 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:31,98 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:31,98 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:10:31,98 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:31,98 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:31,98 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:31,98 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:10:31,98 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:31,98 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:31,294 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 09:10:31,613 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 09:10:31,633 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 09:10:31,953 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 09:10:32,911 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:10:32,912 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:32,912 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:10:32,912 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:10:32,912 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:32,913 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 09:10:32,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 09:10:32,913 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 09:10:32,913 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:33,74 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:33,74 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:10:33,74 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 09:10:33,74 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 09:10:33,74 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:33,74 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 09:10:33,74 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 09:10:33,74 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:33,78 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:33,78 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:33,78 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:10:33,78 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:33,79 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 09:10:33,79 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 09:10:33,80 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:33,80 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica] 09:10:33,82 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:33,82 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596, primaryVersion=13] 09:10:33,83 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596 for shard member-3-shard-cars-config_testAddShardReplica 09:10:33,83 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1169810849] 09:10:33,84 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Shard created, persistent : true 09:10:33,84 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1169810849] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:33,84 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#513138737 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 09:10:33,84 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-3-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 09:10:33,84 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#513138737] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:33,85 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal open: applyTo=0 09:10:33,86 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testAddShardReplica 09:10:33,87 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 09:10:33,87 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:10:33,87 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-3-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:33,88 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-3-shard-cars-config_testAddShardReplica: Recovery completed in in 3.040 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:33,89 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:10:33,89 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 09:10:33,89 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:33,89 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:33,89 AM [cluster-test-shard-dispatcher-10] [INFO] Follower - member-3-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]}}" message is greater than follower's term 0 - updating term 09:10:33,89 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 09:10:33,89 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: snapshot is durable as of 2026-03-18T09:10:33.087516572Z 09:10:33,91 AM [cluster-test-shard-dispatcher-10] [INFO] Follower - member-3-shard-cars-config_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddShardReplica 09:10:33,92 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 09:10:33,92 AM [cluster-test-shard-dispatcher-10] [INFO] SnapshotManager - member-3-shard-cars-config_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:10:33,92 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 09:10:33,92 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3d9a5218 09:10:33,92 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 09:10:33,92 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:10:33,94 AM [cluster-test-shard-dispatcher-10] [INFO] SnapshotManager - member-3-shard-cars-config_testAddShardReplica: snapshot is durable as of 2026-03-18T09:10:33.092324381Z 09:10:33,94 AM [cluster-test-shard-dispatcher-10] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testAddShardReplica: Applying snapshot 09:10:33,94 AM [cluster-test-shard-dispatcher-10] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:10:33,95 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#972352034] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 09:10:33,95 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#972352034] 09:10:33,95 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 446.2 μs 09:10:33,95 AM [cluster-test-shard-dispatcher-10] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testAddShardReplica: Done applying snapshot 09:10:33,95 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 09:10:33,95 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#972352034] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:33,95 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 90.42 μs 09:10:33,96 AM [cluster-test-shard-dispatcher-25] [INFO] AbstractLeader - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 09:10:33,98 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596 09:10:33,98 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 09:10:33,98 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testAddShardReplica 09:10:33,98 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 09:10:33,100 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:33,102 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@39a5d173 09:10:33,102 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 09:10:33,102 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:10:33,102 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1641538917] 09:10:33,102 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 09:10:33,103 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:33,103 AM [cluster-test-pekko.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1169810849] 09:10:33,103 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 09:10:33,103 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1169810849] 09:10:33,103 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:33,103 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-576155259] 09:10:33,103 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:10:33,103 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 09:10:33,103 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 09:10:33,103 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 09:10:33,104 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:33,104 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica] 09:10:33,105 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:33,105 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173, primaryVersion=13] 09:10:33,107 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-786540676] 09:10:33,107 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 09:10:33,107 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-786540676] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:33,108 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#-1381810628 created and ready for shard:member-3-shard-cars-oper_testAddShardReplica 09:10:33,108 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-3-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 09:10:33,108 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#-1381810628] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:33,109 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 09:10:33,110 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173 for shard member-3-shard-cars-oper_testAddShardReplica 09:10:33,111 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-3-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 09:10:33,112 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testAddShardReplica 09:10:33,112 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-3-shard-cars-oper_testAddShardReplica: Recovery completed in in 2.214 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:33,112 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:33,112 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:10:33,112 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:33,112 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testAddShardReplica is initialized 09:10:33,112 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , received role change from null to Follower 09:10:33,112 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 09:10:33,112 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 09:10:33,112 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testAddShardReplica from null to Follower 09:10:33,112 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 09:10:33,113 AM [cluster-test-shard-dispatcher-10] [INFO] Follower - member-3-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]}}" message is greater than follower's term 0 - updating term 09:10:33,115 AM [cluster-test-shard-dispatcher-25] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2026-03-18T09:10:33.112662369Z 09:10:33,115 AM [cluster-test-shard-dispatcher-10] [INFO] Follower - member-3-shard-cars-oper_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddShardReplica 09:10:33,115 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@8a4f9f4 09:10:33,116 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:33,116 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 09:10:33,116 AM [cluster-test-shard-dispatcher-10] [INFO] SnapshotManager - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 09:10:33,118 AM [cluster-test-shard-dispatcher-10] [INFO] SnapshotManager - member-3-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2026-03-18T09:10:33.116303712Z 09:10:33,118 AM [cluster-test-shard-dispatcher-10] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot 09:10:33,118 AM [cluster-test-shard-dispatcher-10] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:10:33,119 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1833651919] with name member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher 09:10:33,119 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1833651919] 09:10:33,119 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: state snapshot applied in 395.3 μs 09:10:33,119 AM [cluster-test-shard-dispatcher-10] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testAddShardReplica: Done applying snapshot 09:10:33,119 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 09:10:33,119 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 70.41 μs 09:10:33,119 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#-1833651919] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:33,120 AM [cluster-test-shard-dispatcher-25] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 09:10:33,121 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173 09:10:33,121 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 09:10:33,122 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:33,123 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testAddShardReplica 09:10:33,123 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 09:10:33,125 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 09:10:33,125 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:33,125 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 09:10:33,125 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-576155259] 09:10:33,125 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6631690 09:10:33,125 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:33,125 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 09:10:33,125 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:10:33,125 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-786540676] 09:10:33,125 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-786540676] 09:10:33,125 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#989578082] 09:10:33,126 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:33,126 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#964720261] 09:10:33,126 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#964720261] 09:10:33,126 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1009562672] 09:10:33,126 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:10:33,126 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#711783292] 09:10:33,126 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#711783292] 09:10:33,170 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#989578082] 09:10:33,170 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:33,170 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 09:10:33,171 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 09:10:33,181 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 09:10:33,194 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state LeaderFrontendState{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=78473, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 09:10:33,202 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:10:33,205 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:10:33,219 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 15.86 ms 09:10:33,259 AM [cluster-test-shard-dispatcher-25] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,285 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: readying transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,285 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 ready 09:10:33,291 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: transitioned member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 to ready 09:10:33,291 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 09:10:33,293 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Validating transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,294 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 validated 09:10:33,294 AM [cluster-test-shard-dispatcher-25] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:10:33,294 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 09:10:33,296 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Preparing transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,300 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 candidate ready 09:10:33,302 AM [cluster-test-shard-dispatcher-25] [DEBUG] UserCohorts - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:10:33,309 AM [cluster-test-shard-dispatcher-25] [DEBUG] UserCohorts - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 09:10:33,309 AM [cluster-test-shard-dispatcher-25] [DEBUG] UserCohorts - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:10:33,309 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 prepared 09:10:33,309 AM [cluster-test-shard-dispatcher-25] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:10:33,310 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 09:10:33,311 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Starting commit for transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,318 AM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 09:10:33,320 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 09:10:33,324 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 09:10:33,326 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Resuming commit of transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-1-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@106b3cc7}, purged=MutableUnsignedLongSet{size=0}} 09:10:33,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:33,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] UserCohorts - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:10:33,328 AM [cluster-test-shard-dispatcher-25] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:10:33,335 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#-254545558] with name member-1-shard-cars-config_testAddShardReplica-DTCL-publisher 09:10:33,335 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#-254545558] 09:10:33,335 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 09:10:33,336 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-config_testAddShardReplica-DTCL-publisher: 122.7 μs 09:10:33,336 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#-254545558] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:33,352 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1641538917] 09:10:33,353 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:33,353 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 09:10:33,353 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 09:10:33,358 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,358 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,361 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5e0bb696}, purged=MutableUnsignedLongSet{size=0}} 09:10:33,361 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@10a12712}, purged=MutableUnsignedLongSet{size=0}} 09:10:33,361 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:33,361 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,361 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:33,361 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,362 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 09:10:33,362 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 09:10:33,362 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 94.57 μs 09:10:33,362 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 66.53 μs 09:10:33,362 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 09:10:33,363 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,373 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 09:10:33,373 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state LeaderFrontendState{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=18171, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 09:10:33,375 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:10:33,375 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:10:33,376 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.715 ms 09:10:33,377 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: replication of member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f52cc9d6438@592758b8 09:10:33,379 AM [cluster-test-shard-dispatcher-25] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: finished purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,381 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:33,381 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,382 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:33,383 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,383 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:33,383 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,383 AM [cluster-test-shard-dispatcher-25] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:33,392 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-576155259] 09:10:33,393 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:33,393 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 09:10:33,393 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 09:10:33,394 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 09:10:33,394 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state LeaderFrontendState{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=10391, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 09:10:33,395 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:10:33,395 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:10:33,397 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 2.391 ms 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: readying transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 ready 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: transitioned member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 to ready 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Validating transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 validated 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Preparing transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 candidate ready 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] UserCohorts - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] UserCohorts - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] UserCohorts - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 prepared 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 09:10:33,399 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Starting commit for transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,400 AM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 233 09:10:33,400 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 09:10:33,402 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Resuming commit of transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,402 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@34d91d0d}, purged=MutableUnsignedLongSet{size=0}} 09:10:33,402 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:33,402 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,402 AM [cluster-test-shard-dispatcher-24] [DEBUG] UserCohorts - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:10:33,402 AM [cluster-test-shard-dispatcher-24] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:10:33,402 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 09:10:33,403 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-522032072] with name member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher 09:10:33,403 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-522032072] 09:10:33,404 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 09:10:33,404 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher: 73.69 μs 09:10:33,404 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#-522032072] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:33,405 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1009562672] 09:10:33,405 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,405 AM [cluster-test-shard-dispatcher-10] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@595dfab4}, purged=MutableUnsignedLongSet{size=0}} 09:10:33,405 AM [cluster-test-shard-dispatcher-10] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:33,405 AM [cluster-test-shard-dispatcher-10] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,406 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 09:10:33,406 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 78.31 μs 09:10:33,406 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,406 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5d9a1153}, purged=MutableUnsignedLongSet{size=0}} 09:10:33,406 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:33,406 AM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,406 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 09:10:33,406 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 68.26 μs 09:10:33,406 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:33,406 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 09:10:33,406 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 09:10:33,407 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 09:10:33,408 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,408 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 09:10:33,408 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state LeaderFrontendState{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=10800, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 09:10:33,409 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:10:33,409 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:10:33,409 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 336.8 μs 09:10:33,410 AM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open snapshot member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,412 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1068642326] 09:10:33,412 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:33,412 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596] 09:10:33,413 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596] 09:10:33,413 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1312307504] 09:10:33,413 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:10:33,413 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:33,413 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:33,464 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:33,516 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:33,567 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:33,617 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:33,669 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:33,720 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:33,771 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:33,822 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:33,873 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:33,874 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007f52cc9d6438@447c972a 09:10:33,874 AM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,874 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:33,874 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,875 AM [cluster-test-shard-dispatcher-10] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:33,876 AM [cluster-test-shard-dispatcher-10] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,876 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:33,876 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 09:10:33,924 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] 09:10:33,924 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#989578082] 09:10:33,925 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:33,925 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#964720261] 09:10:33,925 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#964720261] 09:10:33,925 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1009562672] 09:10:33,925 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:10:33,926 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#711783292] 09:10:33,926 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#711783292] 09:10:33,926 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1641538917] 09:10:33,926 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:33,926 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1169810849] 09:10:33,926 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1169810849] 09:10:33,927 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-576155259] 09:10:33,927 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:10:33,927 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-786540676] 09:10:33,927 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-786540676] 09:10:33,927 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 09:10:33,929 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2557] as [Down] 09:10:33,930 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 09:10:33,930 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 09:10:33,932 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Stopping Shard 09:10:33,933 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal closed 09:10:33,937 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:33,937 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:33,937 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:33,938 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:33,938 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3: Received MemberRemoved: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:33,938 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:33,938 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:10:33,938 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:10:33,938 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 09:10:33,938 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Node has been marked as DOWN. Shutting down myself 09:10:33,938 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Shutting down... 09:10:33,938 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Successfully shut down 09:10:33,938 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ClusterDowningReason] 09:10:33,938 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 09:10:33,939 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1641538917] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1641538917] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:33,939 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Stopping Shard 09:10:33,940 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testAddShardReplica: journal closed 09:10:33,941 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:10:33,943 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:33,943 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:33,947 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:33,947 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:33,948 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:33,949 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:33,950 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:10:33,972 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:33,985 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [8455993014172369317] 09:10:33,985 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.4.0] ... 09:10:33,987 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Registered cluster JMX MBean [pekko:type=Cluster] 09:10:33,987 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:10:33,995 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:33,995 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:34,1 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:34,6 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:34,11 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 09:10:34,14 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1956716009] 09:10:34,14 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:10:34,15 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: applying snapshot ShardManagerSnapshot [ShardList = [cars] ] 09:10:34,16 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: adding shard cars 09:10:34,16 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1956716009] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:34,16 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} started 09:10:34,16 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1956716009] 09:10:34,17 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 09:10:34,17 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testAddShardReplica 09:10:34,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@63110936} 09:10:34,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:34,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testAddShardReplica 09:10:34,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:34,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 09:10:34,18 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1057122706] 09:10:34,18 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Shard created, persistent : true 09:10:34,19 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1057122706] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:34,19 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-3-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 09:10:34,19 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#54630608 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 09:10:34,19 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#54630608] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:34,26 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal open: applyTo=3 09:10:34,28 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: Applying recovered snapshot 09:10:34,34 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:10:34,36 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-1521187790] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 09:10:34,36 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-1521187790] 09:10:34,36 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 2.114 ms 09:10:34,37 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 09:10:34,37 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 147.0 μs 09:10:34,37 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 09:10:34,37 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-1521187790] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:34,42 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3129b9f8}, purged=MutableUnsignedLongSet{size=0}} 09:10:34,42 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:34,42 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:34,42 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 09:10:34,42 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 09:10:34,43 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 09:10:34,43 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-cars-config_testAddShardReplica: Recovery completed in in 16.23 ms: last log index = 3, last log term = 1, napshot index = 0, snapshot term = 1, journal size = 3 09:10:34,43 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:10:34,43 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 09:10:34,43 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:10:34,43 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 09:10:34,43 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:34,43 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 09:10:34,43 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1057122706] 09:10:34,44 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#1057122706] 09:10:34,44 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1956716009] 09:10:34,44 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:10:34,44 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 09:10:34,45 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 09:10:34,45 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 09:10:34,45 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 09:10:34,961 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-3966851390747401946] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:34,961 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-3966851390747401946] but current is [8455993014172369317]. Reason: ActorSystem terminated 09:10:35,39 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] LocalActorRefProvider$$anon$3 - pekkoDeadLetter Message [org.apache.pekko.cluster.ddata.Replicator$Internal$Status] wrapped in [org.apache.pekko.actor.ActorSelectionMessage] to Actor[pekko://cluster-test/] was dropped. Unknown system with UID [-3332200726130069411]. This system with UID [8455993014172369317] was probably restarted. Messages will be accepted when new handshake has been completed.. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:35,49 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] LocalActorRefProvider$$anon$3 - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] wrapped in [org.apache.pekko.actor.ActorSelectionMessage] to Actor[pekko://cluster-test/] was dropped. Unknown system with UID [-2182822994634547345]. This system with UID [8455993014172369317] was probably restarted. Messages will be accepted when new handshake has been completed.. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:35,50 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:35,50 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:35,50 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@69e96246 09:10:35,50 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 09:10:35,51 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 09:10:35,51 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 09:10:35,51 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:35,52 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:10:35,52 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:10:35,152 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:35,452 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:10:35,924 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#575604113]] intended for system uid [-3966851390747401946], self uid is [8455993014172369317] 09:10:36,31 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:10:36,94 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#1663804389]] intended for system uid [-3966851390747401946], self uid is [8455993014172369317] 09:10:37,51 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:10:38,12 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Down)]. 09:10:38,13 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:38,13 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:38,18 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:38,18 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:38,71 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:10:38,112 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#1663804389]] intended for system uid [-3966851390747401946], self uid is [8455993014172369317] 09:10:39,32 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2557] 09:10:39,33 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:39,33 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:39,34 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-3966851390747401946] but current is [8455993014172369317]. Reason: Cluster member removed, previous status [Down] 09:10:39,37 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:39,37 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:39,37 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-3966851390747401946] but current is [8455993014172369317]. Reason: Cluster member removed, previous status [Down] 09:10:39,91 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:10:39,95 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:39,97 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:39,97 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:10:39,97 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:10:39,98 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is ready - notifying 1 OnShardInitialized callbacks 09:10:39,98 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 09:10:39,98 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:10:39,98 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:10:39,98 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 09:10:39,98 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:39,98 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 09:10:39,98 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:10:39,98 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 09:10:39,98 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-2-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 09:10:39,100 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: retiring state LeaderFrontendState{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=5718161510, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 09:10:39,102 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state LeaderFrontendState{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, nanosAgo=32381, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 09:10:39,104 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:10:39,104 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:10:39,104 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 406.9 μs 09:10:39,105 AM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-1-txn-0-1 09:10:39,108 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 09:10:39,108 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 09:10:39,108 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 09:10:39,108 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Stopping Shard 09:10:39,109 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal closed 09:10:39,109 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:10:39,109 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:10:39,111 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:39,111 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:39,115 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:39,115 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:39,115 AM [cluster-test-pekko.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:39,115 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:39,118 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:10:39,130 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 09:10:39,130 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddShardReplica 09:10:39,131 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 09:10:39,131 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Stopping Shard 09:10:39,132 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddShardReplica: journal closed 09:10:39,132 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:10:39,132 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:10:39,132 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 09:10:39,133 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddShardReplica 09:10:39,133 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 09:10:39,133 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#989578082] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#989578082] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:39,133 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#989578082] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#989578082] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:39,133 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Stopping Shard 09:10:39,133 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddShardReplica: journal closed 09:10:39,134 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:10:39,134 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:10:39,135 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:39,135 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:40,52 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:10:40,53 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:40,53 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:40,53 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 09:10:40,53 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:10:40,53 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 09:10:40,53 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 09:10:40,53 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 09:10:40,53 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 09:10:40,53 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-1-shard-cars-oper_testAddShardReplica: Peer address for peer member-3-shard-cars-oper_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 09:10:40,53 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-1-shard-cars-config_testAddShardReplica: Peer address for peer member-3-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 09:10:40,131 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [8455993014172369317] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:40,151 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-3332200726130069411] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:40,154 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:40,155 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:40,158 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:10:40.161] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:40,171 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 09:10:40,171 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplica 09:10:40,172 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 09:10:40,172 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@35498380 09:10:40,172 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@35498380 09:10:40,172 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: pauseLeader successfully completed - doing transfer 09:10:40,173 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@24382f7b 09:10:41,72 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:10:45,142 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 09:10:45,142 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:45,143 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:46,501 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#1977767596] to Actor[pekko://cluster-test/deadLetters] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:46,502 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#909133173] to Actor[pekko://cluster-test/deadLetters] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:46,558 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:10:47,162 AM [cluster-test-pekko.actor.default-dispatcher-12] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 09:10:47,162 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:10:47,162 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:47,162 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:47,840 AM [cluster-test-pekko.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:10:49,431 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: leader transfer aborted 09:10:49,431 AM [cluster-test-shard-dispatcher-24] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: Failed to transfer leadership in 9.259 s 09:10:49,431 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In unpauseLeader 09:10:49,434 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Stopping Shard 09:10:49,435 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplica: journal closed 09:10:49,435 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:10:49,435 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:10:49,436 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 09:10:49,437 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplica 09:10:49,437 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@521532da 09:10:49,437 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@521532da 09:10:49,437 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: pauseLeader successfully completed - doing transfer 09:10:49,437 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 09:10:49,437 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7100fd43 09:10:50,411 AM [cluster-test-shard-dispatcher-24] [WARN] Leader - member-1-shard-cars-oper_testAddShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 09:10:50,411 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 09:10:50,412 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: leader transfer aborted 09:10:50,412 AM [cluster-test-shard-dispatcher-24] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: Failed to transfer leadership in 974.8 ms 09:10:50,412 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In unpauseLeader 09:10:50,412 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:10:50,412 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Leader to IsolatedLeader 09:10:50,412 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 09:10:50,412 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Stopping Shard 09:10:50,413 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplica: journal closed 09:10:50,413 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:10:50,413 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:10:50,415 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:50,415 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:50,418 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:10:50.419] [cluster-test-pekko.actor.internal-dispatcher-5] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:50,447 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:50,460 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-2461016123614668524] 09:10:50,460 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:10:50,463 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:10:50,463 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:10:50,465 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:50,465 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:50,477 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:10:50,477 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:10:50,477 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:50,478 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:10:50,479 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:50,481 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 09:10:50,483 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-978971811] 09:10:50,484 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:10:50,484 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:10:50,484 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:10:50,484 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,484 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,484 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:50,484 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,484 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,484 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-978971811] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,487 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 09:10:50,487 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@5e4f9b8e} 09:10:50,487 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:50,487 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,487 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:50,488 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,488 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,488 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#609566392] 09:10:50,488 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,488 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#609566392] 09:10:50,488 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#609566392] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,488 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 09:10:50,489 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1603684153 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,489 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:50,489 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1603684153] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,489 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 09:10:50,491 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 09:10:50,491 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:10:50,492 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,492 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,492 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-55788017] 09:10:50,492 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-55788017] 09:10:50,492 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 09:10:50,492 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-55788017] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,493 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#381496343] 09:10:50,493 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#1494094313 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,493 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#381496343] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,493 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#1494094313] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,493 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 09:10:50,493 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:10:50,493 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:10:50,493 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:10:50,493 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,493 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,494 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 09:10:50,495 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:10:50,495 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:50,495 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,495 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,497 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed in in 5.504 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:50,497 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,497 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:10:50,497 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,497 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 09:10:50,497 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:10:50,497 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 09:10:50,497 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 09:10:50,498 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:10:50,498 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@72023b1d} 09:10:50,498 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 09:10:50,499 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed in in 4.221 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:50,499 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,499 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,499 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:10:50,499 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 09:10:50,499 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:50,499 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:10:50,499 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,500 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 09:10:50,500 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,500 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,500 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1024933216] 09:10:50,500 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,500 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,500 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,500 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-810872205] 09:10:50,500 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1024933216] 09:10:50,500 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 09:10:50,501 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1024933216] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,501 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 09:10:50,501 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1543412167 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,501 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1543412167] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,501 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-810872205] 09:10:50,501 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-810872205] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,501 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 09:10:50,502 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-40169976 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,502 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 09:10:50,502 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-40169976] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,502 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 09:10:50,503 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 09:10:50,504 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 09:10:50,505 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:10:50,506 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed in in 2.574 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:50,506 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:10:50,506 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:10:50,506 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,506 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 09:10:50,506 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,506 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:10:50,506 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed in in 2.524 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:50,506 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:10:50,506 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:10:50,507 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 09:10:50,507 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 09:10:50,507 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,507 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,507 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:10:50,507 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:10:50,507 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 09:10:50,509 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:50,522 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-5941345540775091499] 09:10:50,522 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.4.0] ... 09:10:50,526 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:50,526 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:10:50,528 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:50,528 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:50,536 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:50,538 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:50,545 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-0-txn-0-0} 09:10:50,545 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 09:10:50,545 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-0} 09:10:50,545 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-1-txn-0-0} 09:10:50,547 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1961584219] 09:10:50,549 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 09:10:50,550 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:50,551 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:50,554 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:10:50,555 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:10:50,555 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 09:10:50,555 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 09:10:50,555 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,555 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,555 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@662e5edf} 09:10:50,555 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:50,555 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,556 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1961584219] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,560 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1357080330] 09:10:50,560 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,560 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 09:10:50,561 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1357080330] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,561 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1701015608] 09:10:50,562 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 09:10:50,562 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 09:10:50,564 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1252940537 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,564 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1252940537] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,565 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1701015608] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,565 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:10:50,565 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 09:10:50,565 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 09:10:50,565 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,566 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,566 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@220d10dd} 09:10:50,566 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:50,566 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,566 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 09:10:50,567 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-727361275] 09:10:50,567 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 09:10:50,567 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 09:10:50,568 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-727361275] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,568 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#788437805 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,568 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#788437805] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,569 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 09:10:50,569 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#883688902] 09:10:50,569 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 09:10:50,569 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#883688902] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,570 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#1943055099 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,570 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#1943055099] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,570 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 09:10:50,570 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:10:50,570 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:10:50,571 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed in in 4.594 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:50,571 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed in in 2.500 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:50,572 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,572 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,572 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:10:50,572 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 09:10:50,572 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:10:50,572 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 09:10:50,572 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:10:50,572 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 09:10:50,574 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,576 AM [cluster-test-shard-dispatcher-17] [INFO] Recovery - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:10:50,576 AM [cluster-test-shard-dispatcher-17] [INFO] Recovery - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed in in 4.016 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:50,577 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,577 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,577 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:10:50,577 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 09:10:50,577 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 09:10:50,577 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1930773697] 09:10:50,577 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,577 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,577 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:10:50,577 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1930773697] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,577 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 09:10:50,577 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 09:10:50,577 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 09:10:50,578 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1509874760 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,578 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1509874760] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,579 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 09:10:50,580 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 09:10:50,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:10:50,580 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 09:10:50,580 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:10:50,581 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 09:10:50,581 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:10:50,582 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed in in 2.127 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:50,582 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:10:50,582 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 09:10:50,582 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,582 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,582 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:10:50,582 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:10:50,582 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 09:10:50,588 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:10:50,615 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-347214821881638225] 09:10:50,616 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.4.0] ... 09:10:50,625 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:50,625 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:10:50,625 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:10:50,627 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:50,628 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:50,628 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:50,628 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,628 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,628 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,628 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,628 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,628 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,628 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,628 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,628 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,628 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,628 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,628 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,630 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:50,630 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:50,632 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:50,633 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:50,638 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 09:10:50,643 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1747790868] 09:10:50,644 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1747790868] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,644 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:10:50,645 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 09:10:50,645 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 09:10:50,645 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,645 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,646 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@73b9bb26} 09:10:50,646 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 09:10:50,647 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:50,647 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,647 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:50,648 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:50,650 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:10:50,653 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,653 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-2034990019] 09:10:50,653 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 09:10:50,654 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-109051779] 09:10:50,654 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1267496079 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,654 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-2034990019] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,654 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:10:50,654 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 09:10:50,654 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 09:10:50,654 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 09:10:50,655 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,655 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 09:10:50,656 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:10:50,656 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 09:10:50,656 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,657 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-842095932] 09:10:50,657 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@399e1aa7} 09:10:50,657 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 09:10:50,657 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:50,657 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,657 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-212156282 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,658 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 09:10:50,658 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:10:50,658 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed in in 2.249 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:50,660 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 09:10:50,661 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:10:50,662 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed in in 2.093 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:50,662 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,662 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,662 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:10:50,662 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:10:50,662 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 09:10:50,662 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:10:50,662 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 09:10:50,664 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-109051779] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,664 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-1267496079] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,664 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-842095932] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,664 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-212156282] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,667 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,667 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 09:10:50,667 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-427077629] 09:10:50,668 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 09:10:50,668 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-427077629] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,668 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,668 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,668 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:10:50,668 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 09:10:50,668 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-82177982] 09:10:50,668 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-82177982] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,668 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1107801732] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,668 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-1107801732 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,668 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 09:10:50,669 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 09:10:50,670 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 09:10:50,671 AM [cluster-test-shard-dispatcher-22] [INFO] Recovery - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:10:50,672 AM [cluster-test-shard-dispatcher-22] [INFO] Recovery - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed in in 1.914 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:50,672 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,672 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,672 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:10:50,672 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1523335872 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,672 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1523335872] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:50,673 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 09:10:50,673 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 09:10:50,675 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 09:10:50,675 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 09:10:50,675 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 09:10:50,676 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed in in 2.011 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:50,676 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,676 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,676 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 09:10:50,679 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:10:50,679 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 09:10:50,679 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:10:50,679 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 09:10:50,680 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:10:50,680 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 09:10:50,680 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 09:10:50,680 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 09:10:50,696 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:10:50,698 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:50,698 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:50,699 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,699 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,699 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,699 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:50,699 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,699 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,699 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,699 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:50,699 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,699 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:50,699 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,699 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:50,699 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:50,837 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 09:10:50,837 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:50,837 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,837 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,837 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 09:10:50,837 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 09:10:50,887 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 09:10:50,887 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 09:10:50,888 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:50,888 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,888 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,888 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:50,888 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,888 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,888 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 09:10:50,888 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 09:10:50,888 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 09:10:50,888 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 09:10:50,918 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 09:10:50,919 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:50,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 09:10:50,919 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:50,919 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 09:10:50,919 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 09:10:51,166 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 09:10:51,247 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 09:10:51,257 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 09:10:51,327 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 09:10:51,485 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:10:51,485 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:10:51,487 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:51,487 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:51,487 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:51,487 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,487 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,488 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,488 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,488 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,488 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:51,488 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,488 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,488 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,488 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,488 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,488 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,490 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,496 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:51,496 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:51,496 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,496 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,496 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,496 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:51,497 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:51,497 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,497 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:51,560 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:51,560 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,560 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,578 AM [cluster-test-shard-dispatcher-26] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 09:10:51,580 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:51,580 AM [cluster-test-shard-dispatcher-19] [INFO] Follower - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:51,590 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:51,590 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:10:51,590 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 09:10:51,590 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 09:10:51,590 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4f763573 09:10:51,590 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 09:10:51,591 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 09:10:51,591 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3759f3e4 09:10:51,592 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 09:10:51,592 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@529b83c3 09:10:51,597 AM [cluster-test-shard-dispatcher-26] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 09:10:51,598 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:51,598 AM [cluster-test-shard-dispatcher-19] [INFO] Follower - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:51,603 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:51,603 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:10:51,604 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 09:10:51,604 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 09:10:51,604 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7cc5db2d 09:10:51,604 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 09:10:51,605 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 09:10:51,605 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c5c77e0 09:10:51,605 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 09:10:51,605 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@70607902 09:10:51,617 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 09:10:51,618 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:51,618 AM [cluster-test-shard-dispatcher-19] [INFO] Follower - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:51,624 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:51,624 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:10:51,624 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 09:10:51,624 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@78fa506d 09:10:51,624 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 09:10:51,624 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 09:10:51,624 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 09:10:51,624 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5863e8b3 09:10:51,625 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 09:10:51,625 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7ea7897e 09:10:51,625 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 09:10:51,625 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 09:10:51,625 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 09:10:51,646 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:10:51,696 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 09:10:51,698 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:51,698 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:10:51,702 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:10:51,702 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 09:10:51,703 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 09:10:51,703 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 09:10:51,703 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3df8d116 09:10:51,703 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 09:10:51,703 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 09:10:51,703 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 09:10:51,703 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 09:10:51,703 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:10:51,703 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 09:10:51,703 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1d617ddb 09:10:51,703 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:10:51,703 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@336671d7 09:10:51,703 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 09:10:51,703 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 09:10:51,703 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 09:10:51,704 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 09:10:51,704 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:10:51,704 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 09:10:51,704 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-978971811] 09:10:51,704 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:51,704 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#609566392] 09:10:51,704 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#609566392] 09:10:51,705 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1961584219] 09:10:51,705 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:51,705 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1357080330] 09:10:51,705 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1357080330] 09:10:51,706 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1747790868] 09:10:51,706 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:51,706 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-2034990019] 09:10:51,706 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-2034990019] 09:10:51,715 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for all shards 09:10:51,715 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testChangeMemberVotingStatesForAllShards 09:10:51,718 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testChangeMemberVotingStatesForAllShards 09:10:51,719 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 09:10:51,718 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 09:10:51,720 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 09:10:51,721 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:10:51,721 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:10:51,721 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 09:10:51,721 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:51,721 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,722 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:10:51,722 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,725 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:51,722 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,724 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:51,723 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 09:10:51,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 09:10:51,728 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 09:10:51,728 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 09:10:51,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 09:10:51,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 09:10:51,728 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:51,729 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:51,729 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,729 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 09:10:51,729 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 09:10:51,729 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 09:10:51,730 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,730 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 09:10:51,730 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 09:10:51,730 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 09:10:51,732 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 09:10:51,733 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 09:10:51,733 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 09:10:51,733 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 09:10:51,733 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-978971811] 09:10:51,733 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 09:10:51,733 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:51,733 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#609566392] 09:10:51,733 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#609566392] 09:10:51,734 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-978971811] 09:10:51,734 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:10:51,734 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-55788017] 09:10:51,734 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-55788017] 09:10:51,734 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#381496343] 09:10:51,734 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:10:51,734 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1024933216] 09:10:51,734 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 09:10:51,734 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 09:10:51,734 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#1024933216] 09:10:51,735 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#381496343] 09:10:51,735 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:10:51,735 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-810872205] 09:10:51,735 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-810872205] 09:10:51,735 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 09:10:51,735 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1961584219] 09:10:51,735 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:51,735 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1357080330] 09:10:51,735 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#-1357080330] 09:10:51,735 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1961584219] 09:10:51,735 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:10:51,735 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-727361275] 09:10:51,735 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-727361275] 09:10:51,736 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1701015608] 09:10:51,736 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:10:51,736 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#883688902] 09:10:51,736 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#883688902] 09:10:51,736 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-1701015608] 09:10:51,736 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 09:10:51,736 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1930773697] 09:10:51,736 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#1930773697] 09:10:51,736 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1747790868] 09:10:51,736 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:51,736 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-2034990019] 09:10:51,736 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-2034990019] 09:10:51,736 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1747790868] 09:10:51,736 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:10:51,736 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-842095932] 09:10:51,736 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-842095932] 09:10:51,736 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-109051779] 09:10:51,736 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:10:51,736 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-427077629] 09:10:51,736 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-427077629] 09:10:51,737 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-109051779] 09:10:51,737 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:10:51,737 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-82177982] 09:10:51,737 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-82177982] 09:10:51,737 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:10:51,737 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,737 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,737 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 09:10:51,738 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 09:10:51,738 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 09:10:51,738 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 09:10:51,739 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 09:10:51,739 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:10:51,739 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:10:51,740 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:10:51,740 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,740 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,740 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:10:51,740 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 09:10:51,741 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 09:10:51,741 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 09:10:51,741 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 09:10:51,741 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:10:51,741 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:10:51,742 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:51,742 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:51,747 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:51,747 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:51,747 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:51,748 AM [cluster-test-pekko.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:51,751 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:10:51,771 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:10:51,772 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:51,772 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:51,772 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:10:51,773 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 09:10:51,773 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 09:10:51,773 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 09:10:51,773 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 09:10:51,773 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:10:51,773 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:10:51,774 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:10:51,774 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:51,774 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:51,774 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:10:51,774 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 09:10:51,774 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 09:10:51,775 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 09:10:51,775 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 09:10:51,775 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:10:51,775 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:10:51,775 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:51,776 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:52,504 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:10:52,754 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-347214821881638225] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:52,795 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-5941345540775091499] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:52,796 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:52,796 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:52,798 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:10:52.801] [cluster-test-pekko.actor.internal-dispatcher-7] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:52,814 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 09:10:52,815 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 09:10:52,815 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 09:10:52,815 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:10:52,815 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@60a33799 09:10:52,816 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@634fb4f4 09:10:52,816 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@56ce225d 09:10:52,816 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@56ce225d 09:10:52,816 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 09:10:52,816 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3d58d333 09:10:52,816 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3d58d333 09:10:52,816 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 09:10:53,194 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 09:10:53,194 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 378.7 ms 09:10:53,194 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 09:10:53,195 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 09:10:53,196 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 09:10:53,294 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 09:10:53,294 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 478.5 ms 09:10:53,294 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 09:10:53,294 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 09:10:53,295 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 09:10:53,295 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:10:53,295 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:10:53,296 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 09:10:53,296 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 09:10:53,296 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 09:10:53,296 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:10:53,296 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5ad2a46 09:10:53,296 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@675151e5 09:10:53,297 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@675151e5 09:10:53,297 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 09:10:53,297 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@59999727 09:10:53,297 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7a486525 09:10:53,297 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@59999727 09:10:53,297 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 09:10:53,824 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 09:10:53,824 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 528.1 ms 09:10:53,825 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 09:10:53,825 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 09:10:53,826 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 09:10:53,854 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 09:10:53,854 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 557.4 ms 09:10:53,854 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 09:10:53,854 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 09:10:53,855 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 09:10:53,855 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:10:53,856 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:10:53,857 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:53,857 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:53,859 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:10:53.861] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:53,891 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:53,903 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-7992953636041141997] 09:10:53,903 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:10:53,911 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:10:53,911 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:10:53,916 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:53,916 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:53,918 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:53,920 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:53,922 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testGetPrefixShardRole 09:10:53,923 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:10:53,923 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:10:53,924 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:10:53,931 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1101871102] 09:10:53,931 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1101871102] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:53,931 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:10:53,932 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:10:53,932 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:10:53,932 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-default-config_testGetPrefixShardRole 09:10:53,932 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:53,932 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 09:10:53,933 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@15348294} 09:10:53,933 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} started 09:10:53,934 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:53,934 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config_testGetPrefixShardRole 09:10:53,934 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:53,935 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 09:10:53,935 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 09:10:53,935 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-default-config_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole on shard member-1-shard-default-config_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1924931830] 09:10:53,935 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1924931830] 09:10:53,935 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Shard created, persistent : true 09:10:53,936 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-default-config_testGetPrefixShardRole: Starting recovery with journal batch size 1 09:10:53,937 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:53,937 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-default-config_testGetPrefixShardRole: journal open: applyTo=0 09:10:53,938 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1924931830] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:53,938 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#1029304626 created and ready for shard:member-1-shard-default-config_testGetPrefixShardRole 09:10:53,938 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#1029304626] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:53,940 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testGetPrefixShardRole 09:10:53,941 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-default-config_testGetPrefixShardRole: Local TermInfo store seeded with TermInfo{term=0} 09:10:53,942 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#-1207991012] 09:10:53,942 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-default-config_testGetPrefixShardRole: Recovery completed in in 4.976 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:53,942 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#-1207991012] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:53,942 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:10:53,942 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:10:53,942 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:10:53,943 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-default-oper_testGetPrefixShardRole 09:10:53,943 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:53,943 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 09:10:53,943 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [default] 09:10:53,943 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-default-config_testGetPrefixShardRole is initialized 09:10:53,943 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 09:10:53,943 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:53,943 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from null to Follower 09:10:53,946 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@49148c12} 09:10:53,946 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} started 09:10:53,946 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:53,946 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-oper_testGetPrefixShardRole 09:10:53,947 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Starting new election term 1 09:10:53,947 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-default-config_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:53,947 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 09:10:53,947 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:53,947 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-default-config_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:10:53,947 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 09:10:53,947 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: became leader with frontend state for [] 09:10:53,947 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Follower to Candidate 09:10:53,947 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Candidate to Leader 09:10:53,947 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole 09:10:53,947 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 09:10:53,947 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 09:10:53,947 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-default-oper_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole on shard member-1-shard-default-oper_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-945417839] 09:10:53,947 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-945417839] 09:10:53,947 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Shard created, persistent : true 09:10:53,947 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-945417839] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:53,947 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@745fbf0 09:10:53,948 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-default-config_testGetPrefixShardRole from Candidate to Leader 09:10:53,948 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testGetPrefixShardRole is ready 09:10:53,948 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-default-oper_testGetPrefixShardRole: Starting recovery with journal batch size 1 09:10:53,948 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#-979799793 created and ready for shard:member-1-shard-default-oper_testGetPrefixShardRole 09:10:53,948 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#-979799793] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:53,949 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-default-oper_testGetPrefixShardRole: journal open: applyTo=0 09:10:53,951 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-default-oper_testGetPrefixShardRole: Local TermInfo store seeded with TermInfo{term=0} 09:10:53,951 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-default-oper_testGetPrefixShardRole: Recovery completed in in 2.102 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:10:53,951 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 09:10:53,951 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:53,951 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [default] 09:10:53,951 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from null to Follower 09:10:53,951 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-default-oper_testGetPrefixShardRole is initialized 09:10:53,953 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Starting new election term 1 09:10:53,953 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-default-oper_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:10:53,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 09:10:53,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:53,953 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Follower to Candidate 09:10:53,953 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-default-oper_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:10:53,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 09:10:53,953 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: became leader with frontend state for [] 09:10:53,953 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Candidate to Leader 09:10:53,953 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole 09:10:53,954 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@10ddb191 09:10:53,954 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-default-oper_testGetPrefixShardRole from Candidate to Leader 09:10:53,954 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testGetPrefixShardRole is ready 09:10:53,997 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1101871102] 09:10:53,997 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : default 09:10:53,997 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#-1924931830] 09:10:53,998 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 09:10:53,998 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-default-config_testGetPrefixShardRole 09:10:53,999 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 09:10:53,999 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@78c2ab0f 09:10:53,999 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-default-config_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@78c2ab0f 09:10:54,0 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Stopping Shard 09:10:54,0 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-default-config_testGetPrefixShardRole: journal closed 09:10:54,1 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:10:54,1 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:10:54,1 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 09:10:54,2 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1101871102] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1101871102] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,2 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007f52cc8403d0] to Actor[pekko://cluster-test/user/$a#1381772657] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#1381772657] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,2 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1101871102] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#1101871102] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,2 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-default-oper_testGetPrefixShardRole 09:10:54,2 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 09:10:54,2 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@7a5ad10f 09:10:54,2 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-default-oper_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@7a5ad10f 09:10:54,3 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Stopping Shard 09:10:54,3 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-default-oper_testGetPrefixShardRole: journal closed 09:10:54,4 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:10:54,4 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:10:54,4 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#-1207991012] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#-1207991012] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,5 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:54,5 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:54,6 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:10:54,115 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:10:54,124 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-8982042005910542647] 09:10:54,124 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:10:54,130 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:10:54,132 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:10:54,132 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:54,132 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:54,142 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:54,144 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:54,145 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:10:54,145 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:10:54,148 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,158 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:10:54,169 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1387452771] 09:10:54,169 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:10:54,169 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:10:54,169 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:10:54,169 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,169 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1387452771] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,170 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,170 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:54,170 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,170 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,172 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@4abd30c2} 09:10:54,172 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 09:10:54,172 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:54,174 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:54,174 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,174 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:54,176 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#284731944] 09:10:54,177 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 09:10:54,177 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#284731944] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,177 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,177 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-729587626 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,177 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 09:10:54,177 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-729587626] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,178 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 09:10:54,179 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,179 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,179 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#284731944] 09:10:54,179 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,180 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,180 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,180 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-118789060] 09:10:54,180 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-118789060] 09:10:54,180 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 09:10:54,180 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1533741581] 09:10:54,180 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 09:10:54,181 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:10:54,181 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:10:54,181 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:10:54,181 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,181 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,181 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 09:10:54,182 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed in in 578.8 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:54,182 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-118789060] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,182 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-80794403 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,183 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1533741581] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,183 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-80794403] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,183 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:54,183 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:54,183 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:10:54,183 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:10:54,183 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:10:54,183 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,183 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:10:54,188 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:54,188 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,188 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed in in 9.261 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:54,188 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,188 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:10:54,188 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:10:54,190 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 09:10:54,192 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@2075c0d1} 09:10:54,192 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:54,192 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,193 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,193 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,193 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1449083530] 09:10:54,193 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,193 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,193 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,193 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1333966146] 09:10:54,193 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1449083530] 09:10:54,193 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 09:10:54,194 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1449083530] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,194 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1859436399 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,194 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1859436399] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,194 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 09:10:54,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:54,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:54,196 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:10:54,196 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,196 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1333966146] 09:10:54,196 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1333966146] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,197 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 09:10:54,197 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:10:54,197 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed in in 697.2 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:54,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:10:54,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:10:54,198 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:54,198 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:54,198 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:10:54,198 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 09:10:54,198 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 09:10:54,198 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1753878346 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,199 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1753878346] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,200 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 09:10:54,201 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,201 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:10:54,203 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:10:54,205 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed in in 4.806 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:54,205 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:10:54,205 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:10:54,205 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:54,205 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:54,205 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:10:54,205 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,207 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:10:54,219 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-3415644205371546603] 09:10:54,219 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.4.0] ... 09:10:54,223 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:54,223 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:10:54,224 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:54,224 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:54,236 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:54,239 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:54,242 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,248 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1259319416] 09:10:54,249 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 09:10:54,250 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:54,252 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:54,256 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,261 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1259319416] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,263 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:10:54,263 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 09:10:54,263 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 09:10:54,263 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@7b107e6b} 09:10:54,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:54,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,265 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-487512876] 09:10:54,265 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 09:10:54,266 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-487512876] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,266 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1886123258 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,266 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1886123258] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,267 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 09:10:54,268 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 09:10:54,273 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 09:10:54,275 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed in in 1.441 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:54,275 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:54,275 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:54,276 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:10:54,276 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,276 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:10:54,276 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:10:54,276 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1113628531] 09:10:54,277 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 09:10:54,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1607645139] 09:10:54,277 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:10:54,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 09:10:54,277 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 09:10:54,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,278 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1113628531] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,278 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1607645139] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,278 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,278 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 09:10:54,278 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@5866dbd4} 09:10:54,278 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:54,278 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,279 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#440762814 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,279 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#440762814] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,279 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 09:10:54,280 AM [cluster-test-shard-dispatcher-20] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed in in 637.5 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:54,280 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:54,280 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:54,281 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:10:54,281 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,281 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#740577135] 09:10:54,281 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#740577135] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,281 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 09:10:54,282 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2673155 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,282 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#2673155] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,282 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 09:10:54,283 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 09:10:54,284 AM [cluster-test-shard-dispatcher-20] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed in in 591.7 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:54,284 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:54,284 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:54,285 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:10:54,287 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:10:54,287 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:10:54,287 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:10:54,288 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,288 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:10:54,294 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,294 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:10:54,294 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:10:54,294 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-383592701] 09:10:54,294 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 09:10:54,295 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 09:10:54,296 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-383592701] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,296 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 09:10:54,296 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1678079190 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,297 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1678079190] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,297 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,297 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:10:54,298 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed in in 585.5 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:54,299 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:10:54,299 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:10:54,299 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:54,299 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:54,299 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:10:54,300 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,300 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:10:54,304 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:10:54,319 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,325 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,325 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:54,336 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-7696609049029786642] 09:10:54,336 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.4.0] ... 09:10:54,338 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:54,338 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:10:54,339 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:54,339 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:54,352 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:54,354 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:54,357 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,358 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-959129803] 09:10:54,363 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 09:10:54,363 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:54,365 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:54,366 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:10:54,366 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 09:10:54,366 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 09:10:54,366 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,367 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,367 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@483f6550} 09:10:54,367 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:54,367 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,369 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,370 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-959129803] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,370 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,370 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-60136872] 09:10:54,371 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 09:10:54,371 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-60136872] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,371 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 09:10:54,371 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#146140344 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,371 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#146140344] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,372 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 09:10:54,373 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed in in 980.6 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:54,374 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:54,374 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:54,374 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1241848987] 09:10:54,374 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 09:10:54,375 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:10:54,375 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 09:10:54,375 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1241848987] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,375 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-574025660] 09:10:54,375 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#466414284 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,375 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:10:54,375 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-574025660] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,375 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#466414284] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,375 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 09:10:54,375 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 09:10:54,375 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,375 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,376 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:10:54,376 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:10:54,376 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 09:10:54,377 AM [cluster-test-shard-dispatcher-10] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed in in 775.4 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:54,377 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:54,377 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:54,377 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:10:54,377 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,379 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:10:54,379 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:10:54,379 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:10:54,379 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,379 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:10:54,382 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@2acf60e5} 09:10:54,383 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 09:10:54,384 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:54,384 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,384 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,384 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#837661858] 09:10:54,384 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1602262385] 09:10:54,384 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 09:10:54,384 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#837661858] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,385 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 09:10:54,385 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1602262385] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,385 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-395301473 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,385 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 09:10:54,385 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1748857927 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,386 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 09:10:54,388 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed in in 645.6 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:54,388 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:54,388 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:54,388 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:10:54,388 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:10:54,389 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:10:54,389 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,389 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:10:54,391 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:10:54,396 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:54,396 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-395301473] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,396 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1748857927] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:54,398 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:54,398 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:54,398 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,398 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,398 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,398 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,398 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,398 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,398 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,398 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,398 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,398 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,398 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,404 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 09:10:54,405 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 09:10:54,405 AM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed in in 576.2 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:54,406 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:10:54,406 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:54,406 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 09:10:54,406 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:54,406 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 09:10:54,406 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,406 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:54,406 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 09:10:55,144 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:10:55,144 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:10:55,144 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:55,144 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:55,144 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,144 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,144 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,144 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,144 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,144 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:55,145 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:55,145 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,145 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,149 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:55,149 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,150 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,150 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,150 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,150 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:55,151 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:55,151 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:55,151 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,151 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,185 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1387452771] 09:10:55,185 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:55,185 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#284731944] 09:10:55,185 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#284731944] 09:10:55,186 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1387452771] 09:10:55,186 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:55,186 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#284731944] 09:10:55,187 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#284731944] 09:10:55,187 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 09:10:55,187 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,187 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,187 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:10:55,187 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:10:55,188 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:10:55,188 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:10:55,188 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:10:55,189 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:10:55,189 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:55,189 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:10:55,190 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,190 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,190 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,190 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,195 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 09:10:55,195 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 09:10:55,195 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:10:55,195 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:55,195 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:55,195 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 09:10:55,195 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 09:10:55,195 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:10:55,195 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:55,195 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:55,195 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 09:10:55,195 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:10:55,195 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 09:10:55,195 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:55,195 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:55,196 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 09:10:55,196 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 09:10:55,196 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 09:10:55,196 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 09:10:55,196 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:10:55,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:10:55,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:55,196 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 09:10:55,196 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 09:10:55,197 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:55,197 AM [cluster-test-shard-dispatcher-19] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:55,197 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:55,197 AM [cluster-test-shard-dispatcher-20] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:55,198 AM [cluster-test-shard-dispatcher-25] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:55,199 AM [cluster-test-shard-dispatcher-10] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:55,199 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:55,199 AM [cluster-test-shard-dispatcher-16] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:55,203 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:10:55,203 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:10:55,204 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:10:55,204 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:10:55,204 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 09:10:55,205 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:10:55,205 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:10:55,205 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:10:55,205 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:10:55,205 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:10:55,206 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 09:10:55,206 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 09:10:55,206 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,206 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 09:10:55,206 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 09:10:55,206 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 09:10:55,206 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,206 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@e2c8587 09:10:55,206 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 09:10:55,206 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 09:10:55,206 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 09:10:55,206 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 09:10:55,206 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3d2e92e4 09:10:55,206 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 09:10:55,206 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 09:10:55,207 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:10:55,207 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:10:55,208 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:10:55,208 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:10:55,208 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@75d79f79 09:10:55,208 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6f027603 09:10:55,208 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:10:55,208 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:10:55,209 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5327ba78 09:10:55,209 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:10:55,209 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@447ecdeb 09:10:55,209 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 09:10:55,209 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6e8324e1 09:10:55,210 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:10:55,210 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@239cfe4f 09:10:55,210 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@64ef264e 09:10:55,210 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:10:55,210 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:10:55,210 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:10:55,211 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:10:55,211 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:10:55,211 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:10:55,212 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 09:10:55,212 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,212 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 09:10:55,212 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 09:10:55,212 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 09:10:55,212 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 09:10:55,213 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:10:55,214 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 09:10:55,214 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:10:55,214 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 09:10:55,214 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 09:10:55,214 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,215 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 09:10:55,215 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@31f52795 09:10:55,215 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 09:10:55,215 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 09:10:55,215 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 09:10:55,215 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1387452771] 09:10:55,215 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:55,215 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#284731944] 09:10:55,215 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#284731944] 09:10:55,216 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1387452771] 09:10:55,216 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:10:55,216 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-118789060] 09:10:55,216 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-118789060] 09:10:55,217 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1533741581] 09:10:55,218 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:55,218 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:10:55,218 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:10:55,218 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:55,218 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:10:55,218 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:10:55,218 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@11a1b5b 09:10:55,218 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:10:55,218 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:10:55,219 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:10:55,218 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:55,219 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:10:55,219 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 09:10:55,219 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 09:10:55,219 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 09:10:55,219 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1449083530] 09:10:55,219 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 09:10:55,219 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:10:55,219 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1449083530] 09:10:55,219 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:10:55,219 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:10:55,219 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 09:10:55,219 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 09:10:55,219 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1533741581] 09:10:55,219 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:10:55,219 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1333966146] 09:10:55,219 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1333966146] 09:10:55,220 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1259319416] 09:10:55,220 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:55,220 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-487512876] 09:10:55,220 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-487512876] 09:10:55,221 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1259319416] 09:10:55,221 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:10:55,221 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1113628531] 09:10:55,221 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1113628531] 09:10:55,223 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 09:10:55,224 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 09:10:55,224 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:10:55,224 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 09:10:55,224 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@62136935 09:10:55,224 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:10:55,224 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 09:10:55,224 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:10:55,224 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 09:10:55,224 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 09:10:55,248 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:55,249 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:55,249 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:10:55,249 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:55,249 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,249 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:10:55,249 AM [cluster-test-shard-dispatcher-10] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,250 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:10:55,250 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:55,250 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,250 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,250 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,250 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,250 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 09:10:55,250 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,250 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,274 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1113628531] 09:10:55,275 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1607645139] 09:10:55,275 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:10:55,275 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#740577135] 09:10:55,275 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#740577135] 09:10:55,276 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1607645139] 09:10:55,276 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 09:10:55,276 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-383592701] 09:10:55,276 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-383592701] 09:10:55,276 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-959129803] 09:10:55,278 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:55,278 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-60136872] 09:10:55,278 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-60136872] 09:10:55,278 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-959129803] 09:10:55,278 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:10:55,278 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1241848987] 09:10:55,278 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#-1241848987] 09:10:55,279 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-574025660] 09:10:55,279 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:10:55,279 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#837661858] 09:10:55,279 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#837661858] 09:10:55,279 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#-574025660] 09:10:55,280 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:10:55,280 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1602262385] 09:10:55,280 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#1602262385] 09:10:55,280 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#1387452771] 09:10:55,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:55,281 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#284731944] 09:10:55,281 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#284731944] 09:10:55,281 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#1533741581] 09:10:55,281 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:10:55,281 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1449083530] 09:10:55,281 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1449083530] 09:10:55,282 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:10:55,282 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,282 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,283 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 09:10:55,283 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 09:10:55,283 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 09:10:55,283 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 09:10:55,284 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 09:10:55,284 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:10:55,284 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:10:55,285 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:10:55,286 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,286 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,286 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:10:55,286 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 09:10:55,286 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 09:10:55,287 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 09:10:55,287 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 09:10:55,287 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:10:55,287 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:10:55,288 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:55,288 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:55,292 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:55,292 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:55,294 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:55,294 AM [cluster-test-pekko.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:55,294 AM [cluster-test-pekko.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:10:55,307 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:10:55,309 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,309 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,309 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:10:55,310 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 09:10:55,310 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 09:10:55,311 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 09:10:55,312 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 09:10:55,312 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:10:55,312 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:10:55,312 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:10:55,312 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,312 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:55,312 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:10:55,313 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 09:10:55,313 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 09:10:55,313 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 09:10:55,313 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 09:10:55,313 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:10:55,313 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:10:55,314 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:55,314 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:56,163 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:10:56,302 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-7696609049029786642] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:56,327 AM [cluster-test-pekko.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:56,327 AM [cluster-test-pekko.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:56,329 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:10:56.331] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:56,332 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-3415644205371546603] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:56,345 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:10:56,346 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:56,346 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 09:10:56,346 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:10:56,347 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6163b0fb 09:10:56,347 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@112e3794 09:10:56,347 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@112e3794 09:10:56,347 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 09:10:56,347 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6fa442a7 09:10:56,347 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@209b5b46 09:10:56,347 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@209b5b46 09:10:56,347 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 09:10:56,812 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 09:10:56,812 AM [cluster-test-shard-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 465.8 ms 09:10:56,812 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 09:10:56,813 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 09:10:56,813 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 09:10:56,822 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 09:10:56,822 AM [cluster-test-shard-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 475.7 ms 09:10:56,822 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 09:10:56,823 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 09:10:56,823 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 09:10:56,823 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:10:56,823 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:10:56,824 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 09:10:56,824 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:56,824 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 09:10:56,824 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:10:56,825 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6b9bdb8b 09:10:56,825 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3740569a 09:10:56,825 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e99489c 09:10:56,825 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3740569a 09:10:56,825 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 09:10:56,825 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1511a169 09:10:56,825 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1511a169 09:10:56,825 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 09:10:57,133 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 09:10:57,133 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 09:10:57,133 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 308.1 ms 09:10:57,133 AM [cluster-test-shard-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 308.3 ms 09:10:57,133 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 09:10:57,133 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 09:10:57,134 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 09:10:57,134 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 09:10:57,135 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 09:10:57,135 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 09:10:57,136 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:10:57,136 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:10:57,137 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:57,137 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:57,140 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:10:57.142] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:57,246 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:10:57,256 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [1902463767684173558] 09:10:57,257 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:10:57,259 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:10:57,259 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:10:57,263 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:57,263 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:57,271 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:10:57,272 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:10:57,272 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:10:57,272 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:57,273 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:57,276 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,277 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#946908226] 09:10:57,278 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:10:57,278 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:10:57,278 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:10:57,278 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,278 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,278 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:57,278 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,278 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,279 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@5beacd70} 09:10:57,280 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#946908226] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,280 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 09:10:57,280 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:57,281 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:57,284 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,284 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1135670118] 09:10:57,285 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:10:57,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:10:57,285 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:10:57,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:57,285 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,287 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1135670118] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,287 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-753371485] 09:10:57,287 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-753371485] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,287 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 09:10:57,288 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1174733599 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,288 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1174733599] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,288 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 09:10:57,289 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 09:10:57,290 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed in in 619.7 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:57,290 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,290 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,290 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:10:57,293 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:57,293 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,293 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,293 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,293 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,293 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-753371485] 09:10:57,293 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,294 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-136452918] 09:10:57,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:10:57,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:10:57,294 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-136452918] 09:10:57,294 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,294 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:10:57,294 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 09:10:57,294 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@24f904be} 09:10:57,295 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 09:10:57,295 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-136452918] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,295 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 09:10:57,296 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 09:10:57,297 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed in in 624.0 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:57,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:10:57,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:10:57,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:57,297 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,297 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1126262244] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,299 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#2082611075] 09:10:57,299 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 09:10:57,299 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1744044948 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,299 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 09:10:57,300 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 09:10:57,301 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed in in 589.6 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:57,301 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,301 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,301 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1126262244 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,301 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:10:57,301 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:10:57,301 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,302 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:10:57,302 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#2082611075] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,302 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1744044948] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,302 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#2082611075] 09:10:57,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,303 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-801960875] 09:10:57,303 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-801960875] 09:10:57,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:10:57,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:10:57,303 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,303 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-801960875] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,303 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 09:10:57,303 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:10:57,304 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1573542603 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,304 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 09:10:57,304 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1573542603] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,305 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 09:10:57,306 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed in in 656.7 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:57,306 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:10:57,306 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:10:57,306 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,306 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,306 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:10:57,306 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,306 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:10:57,310 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:57,324 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [2611525960375479102] 09:10:57,324 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.4.0] ... 09:10:57,327 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:57,327 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:10:57,329 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:57,329 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:57,348 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:57,351 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:57,360 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,364 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1017909410] 09:10:57,365 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1017909410] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,365 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 09:10:57,366 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:57,367 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:57,369 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,370 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:10:57,370 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 09:10:57,370 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 09:10:57,370 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,371 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,371 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@6f5f2a04} 09:10:57,371 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:57,371 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,373 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-2079310225] 09:10:57,374 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 09:10:57,375 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#88323336 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,376 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 09:10:57,379 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,379 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-2079310225] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,379 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#88323336] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,380 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2032878796] 09:10:57,380 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 09:10:57,381 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 09:10:57,383 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 09:10:57,383 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 09:10:57,383 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2032878796] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,383 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 09:10:57,384 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-92786158] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,385 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed in in 1.376 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:57,385 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,385 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,385 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:10:57,385 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:10:57,385 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed in in 1.785 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:57,385 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,385 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,386 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:10:57,386 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1963922173] 09:10:57,388 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:10:57,388 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 09:10:57,388 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 09:10:57,388 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,388 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,389 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@1bd0322a} 09:10:57,389 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:57,389 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,389 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#258104591] 09:10:57,389 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 09:10:57,390 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 09:10:57,390 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 09:10:57,391 AM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed in in 420.8 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:57,391 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,391 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,391 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#2007481298 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,391 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:10:57,391 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,392 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:10:57,392 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:10:57,392 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1460498977] 09:10:57,392 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 09:10:57,392 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#2030902368 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,392 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 09:10:57,393 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 09:10:57,394 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed in in 409.5 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:57,394 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:10:57,394 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:10:57,394 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,394 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:10:57,396 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-92786158 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,396 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:10:57,396 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,396 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1963922173] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,396 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#258104591] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,396 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#2007481298] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,396 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1460498977] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,396 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#2030902368] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,396 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:10:57,396 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:10:57,396 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:10:57,396 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,397 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:10:57,423 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,424 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,424 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:10:57,424 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,425 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:10:57,429 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:10:57,454 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-2032585984654330959] 09:10:57,456 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.4.0] ... 09:10:57,459 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:57,459 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:10:57,469 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:57,470 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:57,472 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:10:57,472 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:10:57,474 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,485 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1258434811] 09:10:57,486 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:10:57,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 09:10:57,486 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 09:10:57,486 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,487 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,487 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1258434811] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,487 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 09:10:57,488 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:57,489 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:10:57,489 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:10:57,491 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:57,491 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@35971cea} 09:10:57,491 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:57,491 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,492 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,494 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,494 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,495 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#873545807] 09:10:57,495 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 09:10:57,496 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 09:10:57,496 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1069430902 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,498 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 09:10:57,499 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#873545807] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,499 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1069430902] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,500 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1990776201] 09:10:57,500 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1990776201] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,500 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 09:10:57,500 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#922669786 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,500 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 09:10:57,500 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#922669786] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,501 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#227577650] 09:10:57,501 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:10:57,501 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 09:10:57,501 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 09:10:57,502 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#227577650] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,508 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed in in 9.939 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:57,509 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,509 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,509 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:10:57,509 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:10:57,509 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 09:10:57,510 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:10:57,510 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed in in 1.063 ms: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:57,510 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,510 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,511 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:10:57,511 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,511 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,511 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,513 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:10:57,513 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:10:57,513 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:10:57,513 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,513 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:10:57,514 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@3bfa5918} 09:10:57,515 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 09:10:57,516 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:10:57,516 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:10:57,516 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,516 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,516 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1570368065] 09:10:57,517 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 09:10:57,517 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1076510877] 09:10:57,517 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 09:10:57,517 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 09:10:57,518 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 09:10:57,518 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 09:10:57,519 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed in in 499.6 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:57,519 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,519 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,519 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 09:10:57,520 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#131998526 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,520 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#902982848 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,520 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:10:57,520 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1570368065] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,520 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1076510877] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,520 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#131998526] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,520 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#902982848] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:10:57,520 AM [cluster-test-shard-dispatcher-20] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed in in 811.5 μs: last log index = 0, last log term = 1, napshot index = -1, snapshot term = -1, journal size = 1 09:10:57,521 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:10:57,521 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:10:57,521 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,521 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:10:57,521 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:10:57,521 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 09:10:57,521 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,521 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,521 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 09:10:57,522 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,522 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 09:10:57,524 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:10:57,526 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:10:57,526 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:57,526 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,526 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,526 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,526 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,526 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,526 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,527 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:10:57,527 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,527 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,527 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,527 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,527 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,527 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:57,626 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 09:10:57,626 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:10:57,626 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,626 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,626 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 09:10:57,626 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 09:10:57,645 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 09:10:57,645 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:10:57,645 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,645 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,645 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 09:10:57,645 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 09:10:57,675 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 09:10:57,675 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:10:57,675 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,675 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,675 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 09:10:57,675 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 09:10:57,705 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 09:10:57,705 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:10:57,705 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 09:10:57,705 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:10:57,705 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 09:10:57,706 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 09:10:57,966 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 09:10:57,995 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 09:10:58,35 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 09:10:58,55 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 09:10:58,274 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:10:58,274 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:10:58,275 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:58,276 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:58,276 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:58,276 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,276 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:58,277 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,277 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,277 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,277 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,277 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,277 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,277 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,282 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,355 AM [cluster-test-shard-dispatcher-29] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 09:10:58,357 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:58,358 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:58,359 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:58,359 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:10:58,359 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:58,359 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:10:58,359 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,359 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,362 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,363 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:10:58,364 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 09:10:58,364 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:10:58,364 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 09:10:58,364 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 09:10:58,364 AM [cluster-test-shard-dispatcher-29] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 09:10:58,365 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7c33cdb6 09:10:58,365 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 09:10:58,365 AM [cluster-test-shard-dispatcher-16] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:58,366 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:10:58,366 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:58,366 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:10:58,366 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@44a78ce3 09:10:58,367 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6bea9b9e 09:10:58,367 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:10:58,368 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:10:58,369 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 09:10:58,369 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:10:58,369 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 09:10:58,369 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 09:10:58,373 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:10:58,373 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 09:10:58,373 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:10:58,373 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 09:10:58,373 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 09:10:58,374 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@72f43c40 09:10:58,374 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 09:10:58,374 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4fc61a14 09:10:58,374 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:10:58,375 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@586ca9b3 09:10:58,375 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:10:58,375 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:10:58,376 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:10:58,378 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 09:10:58,378 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:10:58,378 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 09:10:58,378 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 09:10:58,395 AM [cluster-test-shard-dispatcher-29] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 09:10:58,396 AM [cluster-test-shard-dispatcher-16] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:58,397 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:58,403 AM [cluster-test-shard-dispatcher-29] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:10:58,403 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 09:10:58,403 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:10:58,404 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 09:10:58,404 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 09:10:58,404 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@76cba6c6 09:10:58,404 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 09:10:58,405 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@227ec975 09:10:58,405 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 09:10:58,405 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:10:58,405 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:10:58,406 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7c9d2106 09:10:58,406 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 09:10:58,406 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:10:58,406 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:10:58,408 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 09:10:58,409 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:10:58,409 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 09:10:58,409 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 09:10:58,409 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 09:10:58,436 AM [cluster-test-shard-dispatcher-29] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 09:10:58,437 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:58,438 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 09:10:58,443 AM [cluster-test-shard-dispatcher-29] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:10:58,443 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 09:10:58,443 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:10:58,443 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6f064ab 09:10:58,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 09:10:58,444 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 09:10:58,444 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 09:10:58,444 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:10:58,444 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2cb9334a 09:10:58,444 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 09:10:58,445 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:10:58,446 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1a772c3 09:10:58,446 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 09:10:58,446 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 09:10:58,447 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 09:10:58,447 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 09:10:58,447 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 09:10:58,447 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 09:10:58,447 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 09:10:58,447 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 09:10:58,447 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 09:10:58,447 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:10:58,447 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 09:10:58,447 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#946908226] 09:10:58,448 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:58,448 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-753371485] 09:10:58,448 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-753371485] 09:10:58,449 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 09:10:58,449 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:10:58,449 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:10:58,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:58,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:10:58,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 09:10:58,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 09:10:58,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:10:58,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:10:58,451 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,451 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,452 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,452 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,452 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 09:10:58,452 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,452 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 09:10:58,452 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 09:10:58,452 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 09:10:58,453 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 09:10:58,453 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 09:10:58,453 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,453 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 09:10:58,453 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 09:10:58,453 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 09:10:58,453 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,454 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,454 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 09:10:58,454 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 09:10:58,454 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 09:10:58,456 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#946908226] 09:10:58,456 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:58,456 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-753371485] 09:10:58,457 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-753371485] 09:10:58,457 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#946908226] 09:10:58,457 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 09:10:58,457 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-136452918] 09:10:58,457 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#-136452918] 09:10:58,458 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1135670118] 09:10:58,458 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 09:10:58,458 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#2082611075] 09:10:58,458 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#2082611075] 09:10:58,458 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1135670118] 09:10:58,458 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 09:10:58,458 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-801960875] 09:10:58,458 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-801960875] 09:10:58,458 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1017909410] 09:10:58,458 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:10:58,458 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-2079310225] 09:10:58,459 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-2079310225] 09:10:58,459 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1017909410] 09:10:58,459 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 09:10:58,459 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2032878796] 09:10:58,459 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#2032878796] 09:10:58,459 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1963922173] 09:10:58,460 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 09:10:58,460 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#258104591] 09:10:58,460 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#258104591] 09:10:58,460 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1963922173] 09:10:58,460 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 09:10:58,460 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1460498977] 09:10:58,461 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1460498977] 09:10:58,461 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1258434811] 09:10:58,461 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:10:58,461 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#873545807] 09:10:58,462 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#873545807] 09:10:58,462 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1258434811] 09:10:58,462 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 09:10:58,462 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1990776201] 09:10:58,462 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1990776201] 09:10:58,480 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:10:58,513 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1990776201] 09:10:58,513 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#227577650] 09:10:58,514 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 09:10:58,514 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1570368065] 09:10:58,514 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#1570368065] 09:10:58,514 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#227577650] 09:10:58,514 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 09:10:58,514 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1076510877] 09:10:58,514 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1076510877] 09:10:58,516 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#946908226] 09:10:58,516 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:10:58,516 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-753371485] 09:10:58,516 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-753371485] 09:10:58,517 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:10:58,517 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,518 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,518 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 09:10:58,518 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 09:10:58,518 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 09:10:58,519 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 09:10:58,519 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 09:10:58,519 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:10:58,519 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:10:58,520 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:10:58,521 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,521 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 09:10:58,522 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 09:10:58,521 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,522 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 09:10:58,523 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 09:10:58,523 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:10:58,523 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:10:58,523 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:10:58,525 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:58,525 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:58,530 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:58,530 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:58,530 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:58,530 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:58,532 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:10:58,540 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:10:58,541 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,541 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,541 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:10:58,541 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 09:10:58,542 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 09:10:58,542 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 09:10:58,542 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 09:10:58,542 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:10:58,542 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:10:58,542 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:10:58,543 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,543 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:10:58,543 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:10:58,543 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 09:10:58,543 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 09:10:58,543 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 09:10:58,543 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 09:10:58,544 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:10:58,544 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:10:58,544 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:10:58,544 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:10:59,62 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 09:10:59,62 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 09:10:59,62 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 09:10:59,72 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 09:10:59,294 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:10:59,543 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-2032585984654330959] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:59,562 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:59,562 AM [cluster-test-pekko.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:10:59,563 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [2611525960375479102] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:10:59,564 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:10:59.567] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:10:59,569 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:10:59,570 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:59,570 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 09:10:59,570 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:10:59,570 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4f8e156a 09:10:59,570 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4f8e156a 09:10:59,570 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 09:10:59,571 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@48ca1b89 09:10:59,571 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@48ca1b89 09:10:59,571 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 09:10:59,571 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@482b8871 09:10:59,571 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2340095 09:10:59,962 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 09:10:59,962 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 392.3 ms 09:10:59,962 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 09:10:59,963 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 09:10:59,963 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 09:11:00,43 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 09:11:00,43 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 472.8 ms 09:11:00,43 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 09:11:00,44 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 09:11:00,45 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 09:11:00,45 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:11:00,45 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:11:00,45 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 09:11:00,46 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:11:00,46 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 09:11:00,46 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:11:00,46 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@33c9367b 09:11:00,46 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@517503c9 09:11:00,47 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5a8610c3 09:11:00,47 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5a8610c3 09:11:00,47 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 09:11:00,47 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c463be9 09:11:00,47 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c463be9 09:11:00,47 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 09:11:00,612 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 09:11:00,613 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 566.4 ms 09:11:00,613 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 09:11:00,614 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 09:11:00,615 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 09:11:00,643 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 09:11:00,643 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 596.8 ms 09:11:00,643 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 09:11:00,644 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 09:11:00,644 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 09:11:00,645 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:11:00,645 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:11:00,646 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:11:00,646 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:11:00,649 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:11:00.651] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:11:00,672 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:11:00,685 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-8832943346299712461] 09:11:00,686 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:11:00,691 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:11:00,691 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:11:00,692 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:11:00,692 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:11:00,705 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:00,706 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:11:00,706 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:11:00,706 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:11:00,710 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:00,713 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 09:11:00,716 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-738963640] 09:11:00,716 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-738963640] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,718 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:11:00,718 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:11:00,718 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:11:00,718 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,719 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:00,719 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:00,719 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,719 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:00,725 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@2c154201} 09:11:00,725 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 09:11:00,725 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:00,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,726 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:00,726 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-1-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-1750802177] 09:11:00,726 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:00,726 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-1750802177] 09:11:00,727 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-1750802177] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,727 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:00,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:00,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-1-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#1835638105] 09:11:00,727 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#1835638105] 09:11:00,727 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 09:11:00,727 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:00,728 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 09:11:00,729 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#1084947036 created and ready for shard:member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:00,729 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#1835638105] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,729 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#1084947036] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,730 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 09:11:00,731 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 09:11:00,732 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-2038857289] 09:11:00,733 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-2038857289] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,734 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:11:00,734 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:11:00,734 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:11:00,734 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,734 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,734 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:00,734 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,734 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,727 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 09:11:00,735 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 09:11:00,736 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 09:11:00,736 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testRemoveShardLeaderReplica: Recovery completed in in 6.279 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:00,736 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:00,736 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:00,737 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:11:00,737 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardLeaderReplica is initialized 09:11:00,737 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 09:11:00,738 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 09:11:00,738 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#1678435494] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,738 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 09:11:00,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@4bd30d39} 09:11:00,738 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 09:11:00,738 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from null to Follower 09:11:00,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:00,738 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,739 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-1-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#-296973060] 09:11:00,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,739 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-1-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#-1447102006] 09:11:00,739 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#-296973060] 09:11:00,739 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 09:11:00,740 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1727674071 created and ready for shard:member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,740 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 09:11:00,741 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 09:11:00,742 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#-1447102006] 09:11:00,742 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 09:11:00,742 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#-778658311 created and ready for shard:member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,742 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 09:11:00,743 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 09:11:00,746 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#-296973060] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,746 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1727674071] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,746 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#-1447102006] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,746 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#-778658311] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,748 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#1678435494 created and ready for shard:member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,751 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:11:00,753 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 09:11:00,754 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed in in 12.04 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:00,754 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:11:00,754 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is initialized 09:11:00,754 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:00,754 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:00,754 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 09:11:00,754 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 09:11:00,754 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 09:11:00,754 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed in in 16.97 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:00,755 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-1-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 09:11:00,755 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-1-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed in in 11.54 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:00,755 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:00,755 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:00,755 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 09:11:00,755 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:11:00,755 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardLeaderReplica is initialized 09:11:00,755 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 09:11:00,755 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 09:11:00,755 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 09:11:00,755 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:00,755 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:00,755 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:11:00,755 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardLeaderReplica is initialized 09:11:00,755 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 09:11:00,755 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 09:11:00,755 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 09:11:00,767 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [2004824932013266747] 09:11:00,767 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.4.0] ... 09:11:00,770 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:11:00,770 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:11:00,771 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:11:00,771 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:11:00,782 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:00,785 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:00,788 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 09:11:00,792 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1900220798] 09:11:00,792 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:11:00,793 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 09:11:00,793 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 09:11:00,793 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,793 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardLeaderReplica 09:11:00,793 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1900220798] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,796 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@436a4468} 09:11:00,797 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 09:11:00,797 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:00,797 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,798 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:00,800 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:00,804 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 09:11:00,807 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardLeaderReplica 09:11:00,808 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-79060890] 09:11:00,808 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-79060890] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,808 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-1190842301] 09:11:00,808 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 09:11:00,808 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-1190842301] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,809 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:11:00,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 09:11:00,809 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 09:11:00,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica#-814305881] 09:11:00,809 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica#-814305881] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,810 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#998841072 created and ready for shard:member-2-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,810 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#998841072] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,810 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 09:11:00,810 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 09:11:00,810 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#517169599 created and ready for shard:member-2-shard-people-config_testRemoveShardLeaderReplica 09:11:00,811 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 09:11:00,812 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 09:11:00,813 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 09:11:00,813 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@4dada275} 09:11:00,814 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#517169599] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,814 AM [cluster-test-shard-dispatcher-22] [INFO] Recovery - member-2-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 09:11:00,814 AM [cluster-test-shard-dispatcher-22] [INFO] Recovery - member-2-shard-people-config_testRemoveShardLeaderReplica: Recovery completed in in 2.711 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:00,815 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:11:00,815 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardLeaderReplica is initialized 09:11:00,815 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:00,815 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:00,815 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 09:11:00,815 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 09:11:00,815 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardLeaderReplica from null to Follower 09:11:00,818 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-2-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 09:11:00,818 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 09:11:00,818 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:00,818 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,819 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,819 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica#-515771736] 09:11:00,819 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica#-515771736] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,819 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 09:11:00,819 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica#2019803580] 09:11:00,819 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica#2019803580] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,819 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 09:11:00,820 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 09:11:00,820 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1267077189 created and ready for shard:member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,820 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1267077189] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,820 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#177544514 created and ready for shard:member-2-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,821 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 09:11:00,822 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 09:11:00,822 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-2-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed in in 8.496 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:00,822 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:00,822 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:00,822 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:11:00,822 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardLeaderReplica is initialized 09:11:00,822 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 09:11:00,823 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 09:11:00,823 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 09:11:00,823 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 09:11:00,823 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#177544514] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,825 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 09:11:00,825 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed in in 4.168 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:00,826 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:00,826 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:00,826 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 09:11:00,826 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:11:00,826 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is initialized 09:11:00,826 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 09:11:00,826 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 09:11:00,826 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-2-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 09:11:00,827 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-2-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed in in 3.826 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:00,827 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:11:00,827 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardLeaderReplica is initialized 09:11:00,832 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:00,832 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:00,834 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 09:11:00,834 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 09:11:00,834 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 09:11:00,841 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:11:00,844 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:11:00,846 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:00,846 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:00,846 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,846 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,846 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,846 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,846 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,846 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:00,846 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,846 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:11:00,846 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,846 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,846 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:00,846 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,847 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:11:00,859 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-555478363433576565] 09:11:00,860 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.4.0] ... 09:11:00,862 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:11:00,862 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:11:00,867 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:11:00,871 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:11:00,876 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:00,877 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:00,880 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 09:11:00,881 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1836863781] 09:11:00,882 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:11:00,882 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 09:11:00,882 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 09:11:00,882 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,882 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardLeaderReplica 09:11:00,882 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1836863781] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,888 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@1283ed2d} 09:11:00,888 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 09:11:00,889 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:00,889 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,889 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:00,890 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:00,893 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardLeaderReplica 09:11:00,893 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-607039307] 09:11:00,893 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 09:11:00,894 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1740240292 created and ready for shard:member-3-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,895 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-607039307] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,895 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1740240292] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,897 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-3-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 09:11:00,897 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:11:00,898 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 09:11:00,899 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:11:00,901 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 09:11:00,901 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:00,901 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,901 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#1521653662] 09:11:00,901 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:00,901 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:11:00,902 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 09:11:00,902 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#1521653662] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,902 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-3-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 09:11:00,902 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#554846578 created and ready for shard:member-3-shard-people-config_testRemoveShardLeaderReplica 09:11:00,902 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-200678182] 09:11:00,903 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#554846578] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,903 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:11:00,903 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 09:11:00,903 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 09:11:00,903 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,903 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,904 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:00,904 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,904 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,906 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-200678182] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,907 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 09:11:00,908 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed in in 9.410 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:00,908 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:11:00,908 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardLeaderReplica is initialized 09:11:00,908 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@1c44110d} 09:11:00,908 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 09:11:00,909 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:00,909 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,909 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 09:11:00,909 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:00,909 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:00,909 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 09:11:00,909 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:00,909 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 09:11:00,909 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#-1553169819] 09:11:00,909 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 09:11:00,909 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#-1553169819] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,909 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 09:11:00,910 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#11597141 created and ready for shard:member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,910 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#11597141] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,911 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 09:11:00,911 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 09:11:00,912 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 09:11:00,912 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,912 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,912 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-3-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#-1553169819] 09:11:00,912 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,913 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,913 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-3-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#403026667] 09:11:00,913 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#403026667] 09:11:00,913 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#403026667] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,913 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 09:11:00,913 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 09:11:00,913 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#1459576972 created and ready for shard:member-3-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,913 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#1459576972] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:00,914 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 09:11:00,917 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-3-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 09:11:00,917 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-3-shard-people-config_testRemoveShardLeaderReplica: Recovery completed in in 6.436 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:00,918 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:11:00,918 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardLeaderReplica is initialized 09:11:00,918 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:00,918 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:00,918 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 09:11:00,918 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:00,918 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 09:11:00,918 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardLeaderReplica from null to Follower 09:11:00,918 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 09:11:00,919 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed in in 6.898 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:00,919 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:00,919 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:00,919 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 09:11:00,919 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:00,919 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:11:00,919 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is initialized 09:11:00,919 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 09:11:00,919 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 09:11:00,919 AM [cluster-test-shard-dispatcher-27] [INFO] Recovery - member-3-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 09:11:00,919 AM [cluster-test-shard-dispatcher-27] [INFO] Recovery - member-3-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed in in 5.217 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:00,919 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:11:00,920 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardLeaderReplica is initialized 09:11:00,920 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:00,920 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:00,920 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 09:11:00,920 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:00,920 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 09:11:00,920 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 09:11:01,76 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 09:11:01,76 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:11:01,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:01,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:01,76 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 09:11:01,76 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Follower to Candidate 09:11:01,155 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 09:11:01,156 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:11:01,156 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:01,156 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:01,156 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 09:11:01,156 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Follower to Candidate 09:11:01,166 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 09:11:01,166 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:11:01,166 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:01,166 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:01,167 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 09:11:01,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 09:11:01,180 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 09:11:01,180 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:11:01,180 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:01,180 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:01,180 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 09:11:01,180 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Follower to Candidate 09:11:01,473 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 09:11:01,533 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 09:11:01,549 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 09:11:01,577 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 09:11:01,709 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:11:01,710 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:11:01,711 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:11:01,720 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 09:11:01,801 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:11:01,872 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 09:11:01,874 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:01,874 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:01,876 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:11:01,895 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:11:01,895 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:11:01,895 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 09:11:01,895 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 09:11:01,895 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4d4cac9b 09:11:01,895 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Candidate to Leader 09:11:01,897 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardLeaderReplica status sync done false 09:11:01,897 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@f4c2d40 09:11:01,897 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardLeaderReplica status sync done false 09:11:01,897 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@777defae 09:11:01,936 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 09:11:01,937 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:01,938 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:01,948 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:11:01,948 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:11:01,948 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 09:11:01,948 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 09:11:01,948 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@66ab31a0 09:11:01,948 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Candidate to Leader 09:11:01,949 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done false 09:11:01,950 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@18bd2d9 09:11:01,950 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardLeaderReplica status sync done false 09:11:01,950 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@79ba0f31 09:11:01,953 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 09:11:01,955 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:01,955 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:01,966 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:11:01,966 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:11:01,966 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 09:11:01,966 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 09:11:01,967 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@438b4b42 09:11:01,967 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Candidate to Leader 09:11:01,967 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 09:11:01,967 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5b44630 09:11:01,968 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7d8584de 09:11:01,968 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 09:11:01,968 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 09:11:01,968 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 09:11:01,969 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 09:11:01,982 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 09:11:01,984 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:01,985 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:01,995 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:11:01,995 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:11:01,995 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5a5710d6 09:11:01,995 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 09:11:01,995 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 09:11:01,995 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 09:11:01,995 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 09:11:01,995 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} is now ready 09:11:01,995 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-738963640] 09:11:01,996 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:11:01,996 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-1750802177] 09:11:01,996 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-1750802177] 09:11:01,996 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1900220798] 09:11:01,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:11:01,997 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-79060890] 09:11:01,997 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-79060890] 09:11:01,997 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2c348d1d 09:11:01,997 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 09:11:01,997 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 09:11:01,997 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1836863781] 09:11:01,997 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:11:01,997 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-607039307] 09:11:01,997 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@187b865d 09:11:01,998 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-607039307] 09:11:01,998 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 09:11:01,998 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 09:11:01,998 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-1}, datastoreType Config 09:11:01,998 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-1}] 09:11:01,999 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:11:01,999 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-1750802177, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1a44f4e4] 09:11:01,999 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-1750802177 for shard member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:02,0 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardLeaderReplica} for shard cars 09:11:02,0 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 09:11:02,0 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 09:11:02,1 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1900220798] 09:11:02,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:11:02,1 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-79060890] 09:11:02,1 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-79060890] 09:11:02,6 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]} has been successfully replicated to a majority of followers 09:11:02,7 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-1750802177] with 10000 ms timeout 09:11:02,7 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1 : Local Shard replica for shard cars has been removed 09:11:02,7 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: persisting the shard list [people] 09:11:02,7 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@47b4c7eb 09:11:02,7 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@47b4c7eb 09:11:02,7 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 09:11:02,10 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2bdd211 09:11:02,10 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 09:11:02,10 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:02,10 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@40b6d8e2 09:11:02,11 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 09:11:02,11 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5d791b36 09:11:02,11 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 09:11:02,15 AM [cluster-test-shard-dispatcher-26] [INFO] Candidate - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 4 09:11:02,15 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:11:02,15 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:02,15 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:02,15 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@48722c24 09:11:02,15 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 09:11:02,15 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 09:11:02,16 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3}" message is greater than follower's term 3 - updating term 09:11:02,21 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:11:02,21 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:11:02,21 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 09:11:02,21 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 09:11:02,21 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2efecb99 09:11:02,21 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 09:11:02,21 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 09:11:02,22 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@509e6f31 09:11:02,22 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 09:11:02,22 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 09:11:02,24 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 09:11:02,56 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-79060890] 09:11:02,56 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1900220798] 09:11:02,57 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:11:02,57 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-79060890] 09:11:02,57 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-79060890] 09:11:02,57 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1836863781] 09:11:02,57 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:11:02,57 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-607039307] 09:11:02,57 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-607039307] 09:11:02,58 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-738963640] 09:11:02,58 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:11:02,58 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Local shard cars not found - shards present: [people] 09:11:02,58 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 09:11:02,58 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testRemoveShardLeaderReplica 09:11:02,59 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardLeaderReplica 09:11:02,59 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 09:11:02,59 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 09:11:02,59 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 09:11:02,60 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardLeaderReplica: journal closed 09:11:02,60 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardLeaderReplica: journal closed 09:11:02,60 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:11:02,60 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:11:02,60 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 09:11:02,61 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 09:11:02,61 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardLeaderReplica 09:11:02,61 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:11:02,61 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 09:11:02,61 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 09:11:02,62 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 09:11:02,62 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardLeaderReplica: journal closed 09:11:02,62 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:11:02,62 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:11:02,63 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:11:02,63 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:11:02,70 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:11:02,70 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:11:02,70 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:11:02,70 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:11:02,72 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:11:02,87 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 09:11:02,87 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardLeaderReplica 09:11:02,87 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardLeaderReplica 09:11:02,88 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:11:02,88 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1a4bb9ab 09:11:02,88 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1a4bb9ab 09:11:02,88 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 09:11:02,88 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@34c940df 09:11:02,88 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 09:11:02,89 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardLeaderReplica: journal closed 09:11:02,261 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done true 09:11:02,262 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 310, lastApplied : -1, commitIndex : -1 09:11:02,281 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done true 09:11:02,281 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 312, lastApplied : -1, commitIndex : -1 09:11:02,581 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:02,601 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:02,632 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cannot append entries because sender's term 3 is less than 4 09:11:02,633 AM [cluster-test-shard-dispatcher-13] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 4 in "AppendEntriesReply{term=4, success=false, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}" message is greater than leader's term 3 - switching to Follower 09:11:02,636 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 4 09:11:02,636 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 09:11:02,636 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:02,636 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 09:11:02,636 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2cc4dec4 09:11:02,636 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 09:11:02,636 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 09:11:02,730 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:11:02,901 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:02,922 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:11:03,79 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-555478363433576565] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:11:03,83 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-555478363433576565] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:11:03,222 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:03,241 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:11:03,542 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:03,563 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 09:11:03,862 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:03,881 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 09:11:04,29 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader not elected in time 09:11:04,29 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to null in 2.022 s 09:11:04,30 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 09:11:04,30 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardLeaderReplica: journal closed 09:11:04,31 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#-1750802177] 09:11:04,181 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:04,201 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:04,502 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:04,522 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:04,822 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:04,842 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:05,141 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:05,161 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:11:05,462 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:05,482 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:05,781 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:11:05,801 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:11:06,101 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:06,120 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:06,422 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:06,441 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:06,742 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:06,762 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:07,10 AM [cluster-test-shard-dispatcher-15] [WARN] Leader - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:11:07,10 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:11:07,10 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:11:07,10 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 09:11:07,11 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:11:07,60 AM [cluster-test-shard-dispatcher-15] [WARN] Leader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:11:07,60 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:11:07,60 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:11:07,60 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 09:11:07,60 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:11:07,62 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:07,62 AM [cluster-test-shard-dispatcher-15] [INFO] IsolatedLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica: IsolatedLeader member-1-shard-people-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 09:11:07,63 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 09:11:07,63 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:11:07,63 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 09:11:07,63 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 09:11:07,63 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 09:11:07,80 AM [cluster-test-shard-dispatcher-15] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:11:07,80 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:11:07,80 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:11:07,80 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 09:11:07,81 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:11:07,82 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:07,82 AM [cluster-test-shard-dispatcher-15] [INFO] IsolatedLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica: IsolatedLeader member-1-shard-cars-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 09:11:07,83 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 09:11:07,83 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:11:07,83 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 09:11:07,83 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 09:11:07,83 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 09:11:07,381 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 09:11:07,401 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 09:11:07,701 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:07,721 AM [cluster-test-shard-dispatcher-15] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:07,812 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 09:11:07,812 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:11:07,812 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:07,812 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:07,816 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:11:07,836 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:11:07,905 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 09:11:07,905 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:07,905 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:08,23 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:11:08,31 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 310, lastApplied : -1, commitIndex : -1 09:11:08,341 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:11:08,352 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:08,661 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:08,671 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:11:08,982 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:08,991 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:09,302 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:09,311 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:09,621 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:11:09,631 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:09,699 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:11:09,941 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:09,950 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:10,262 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:10,270 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:10,581 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 09:11:10,590 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:10,902 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 09:11:10,912 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 09:11:11,221 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:11:11,230 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 09:11:11,293 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer aborted 09:11:11,293 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: Failed to transfer leadership in 9.205 s 09:11:11,293 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In unpauseLeader 09:11:11,294 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 09:11:11,294 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardLeaderReplica: journal closed 09:11:11,295 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:11:11,295 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:11:11,295 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 09:11:11,296 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 09:11:11,296 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardLeaderReplica 09:11:11,296 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:11:11,296 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 09:11:11,297 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 09:11:11,297 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 09:11:11,297 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardLeaderReplica: journal closed 09:11:11,298 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:11:11,298 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:11:11,299 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:11:11,300 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:11:12,180 AM [cluster-test-shard-dispatcher-13] [WARN] Leader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:11:12,180 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:11:12,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:11:12,180 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 09:11:12,180 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:11:12,189 AM [cluster-test-shard-dispatcher-13] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:11:12,189 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 09:11:12,189 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 09:11:12,189 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 09:11:12,190 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 09:11:12,318 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:11:12,318 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:11:12,319 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [2004824932013266747] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:11:12,320 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:11:12.321] [cluster-test-pekko.actor.internal-dispatcher-5] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:11:12,334 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 09:11:12,334 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardLeaderReplica 09:11:12,334 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 09:11:12,335 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 09:11:12,335 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardLeaderReplica: journal closed 09:11:12,335 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:11:12,335 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:11:12,336 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 09:11:12,336 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 09:11:12,336 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardLeaderReplica 09:11:12,336 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:11:12,336 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 09:11:12,336 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 09:11:12,336 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardLeaderReplica: journal closed 09:11:12,336 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 09:11:12,337 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:11:12,337 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:11:12,338 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:11:12,338 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:11:12,340 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:11:12.341] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:11:12,365 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:11:12,372 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [4797862553173102312] 09:11:12,372 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:11:12,375 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:11:12,375 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:11:12,376 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:11:12,376 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:11:12,386 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,388 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,390 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,393 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-1189144609] 09:11:12,394 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:11:12,394 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:11:12,394 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:11:12,395 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,395 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,395 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:12,395 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,395 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,395 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:11:12,395 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:11:12,395 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:11:12,396 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-1189144609] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,398 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@72238180} 09:11:12,399 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 09:11:12,399 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:12,399 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,399 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,400 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,402 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,403 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,403 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,403 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#323156822] 09:11:12,403 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,403 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,403 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard#956808671] 09:11:12,403 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,403 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-439395243] 09:11:12,403 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#323156822] 09:11:12,403 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard#956808671] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,403 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:11:12,403 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#323156822] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,403 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 09:11:12,403 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:11:12,403 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:11:12,404 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,404 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,404 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-439395243] 09:11:12,404 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-439395243] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,404 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 09:11:12,404 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 09:11:12,406 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 09:11:12,407 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#1670258138 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,407 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:12,407 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#810443486] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,407 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,407 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,409 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 09:11:12,410 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#810443486 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,410 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#1670258138] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,410 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 09:11:12,410 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,411 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed in in 4.816 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,411 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,411 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,412 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 09:11:12,412 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,412 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:11:12,412 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 09:11:12,413 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@62651750} 09:11:12,413 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed in in 2.818 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,413 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,413 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,413 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:11:12,413 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 09:11:12,414 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 09:11:12,414 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 09:11:12,414 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:11:12,414 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,414 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,414 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 09:11:12,414 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:11:12,414 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 09:11:12,414 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 09:11:12,414 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 09:11:12,414 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,414 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@42a8c9ed 09:11:12,414 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 09:11:12,415 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 09:11:12,415 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:11:12,415 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,415 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,415 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:11:12,415 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 09:11:12,415 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 09:11:12,415 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 09:11:12,415 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 09:11:12,415 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,415 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1a3cbf7e 09:11:12,415 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 09:11:12,415 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForSingleNodeShard is ready 09:11:12,416 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 09:11:12,416 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:12,416 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,416 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 09:11:12,416 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} is now ready 09:11:12,416 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,416 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,416 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-409746427] 09:11:12,416 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,416 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,417 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,417 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#1108721987] 09:11:12,417 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-409746427] 09:11:12,417 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 09:11:12,417 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 09:11:12,418 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#1108721987] 09:11:12,418 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 09:11:12,418 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 09:11:12,419 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 09:11:12,419 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-409746427] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,419 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#1108721987] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,419 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#192890537 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,419 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#845536234 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,419 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#192890537] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,419 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#845536234] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,420 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 09:11:12,420 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,420 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard ShardName{value=cars}: {member-1=false} 09:11:12,420 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-1=false}] 09:11:12,421 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed in in 2.217 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,421 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:11:12,421 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 09:11:12,421 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:11:12,421 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,421 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,421 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,421 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 09:11:12,422 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed in in 1.669 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,422 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:11:12,422 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 09:11:12,422 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,422 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,422 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 09:11:12,422 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,423 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=INVALID_REQUEST, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,423 AM [cluster-test-pekko.actor.default-dispatcher-12] [ERROR] ClusterAdminRpcService - Failed to change member voting states for shard ShardName{value=cars} java.lang.IllegalArgumentException: The requested voting state change for shard cars is invalid. At least one member must be voting at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.lambda$changeShardMembersVotingStatus$17(ShardManager.java:1508) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482) at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59) at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:62) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 09:11:12,423 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#-1189144609] 09:11:12,423 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 09:11:12,423 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:11:12,423 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,423 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,423 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 09:11:12,423 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:11:12,424 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 09:11:12,424 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 09:11:12,424 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:11:12,424 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,424 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,424 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 09:11:12,424 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 09:11:12,424 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 09:11:12,424 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:11:12,424 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 09:11:12,424 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 09:11:12,424 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 09:11:12,424 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:11:12,424 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,424 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#323156822] 09:11:12,424 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@594c77c9 09:11:12,424 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#323156822] 09:11:12,424 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,424 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 09:11:12,424 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@578b9290 09:11:12,424 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 09:11:12,424 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForSingleNodeShard is ready 09:11:12,425 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 09:11:12,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,425 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:11:12,425 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@53b41cb9 09:11:12,425 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@53b41cb9 09:11:12,425 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 09:11:12,427 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: journal closed 09:11:12,427 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5bc4e276 09:11:12,427 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5bc4e276 09:11:12,427 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 09:11:12,427 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: journal closed 09:11:12,428 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:11:12,428 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:11:12,428 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 09:11:12,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 09:11:12,428 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:11:12,429 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@16b698ab 09:11:12,429 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@16b698ab 09:11:12,429 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 09:11:12,429 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: journal closed 09:11:12,429 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5503177e 09:11:12,430 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5503177e 09:11:12,430 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 09:11:12,430 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: journal closed 09:11:12,431 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:11:12,431 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:11:12,432 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:11:12,432 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:11:12,433 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:11:12,452 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:11:12,460 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [1033513772644554273] 09:11:12,460 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.4.0] ... 09:11:12,463 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 09:11:12,463 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 09:11:12,464 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:11:12,464 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:11:12,474 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:11:12,474 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:11:12,474 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 09:11:12,474 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,476 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,477 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 09:11:12,480 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1633215497] 09:11:12,480 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1633215497] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,480 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 09:11:12,480 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 09:11:12,480 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 09:11:12,480 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,481 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,481 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 09:11:12,481 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:12,481 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,481 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,481 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,481 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@5bc6e78e} 09:11:12,481 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:12,482 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,482 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,484 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 09:11:12,485 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1116485252] 09:11:12,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,485 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-157318849] 09:11:12,485 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,485 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard on shard member-1-shard-people-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#1167934086] 09:11:12,485 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-157318849] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,486 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 09:11:12,486 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 09:11:12,486 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 09:11:12,486 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,486 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,486 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#1167934086] 09:11:12,486 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1116485252] 09:11:12,486 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#1167934086] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,486 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 09:11:12,486 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1116485252] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,486 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 09:11:12,487 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#212413665 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,487 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 09:11:12,487 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#212413665] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,487 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1583063589 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,487 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1583063589] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,488 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 09:11:12,488 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 09:11:12,489 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 09:11:12,490 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:12,490 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,490 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,491 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,491 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed in in 3.241 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,492 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,492 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,492 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,492 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 09:11:12,492 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is initialized 09:11:12,492 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed in in 3.278 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,492 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 09:11:12,492 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 09:11:12,492 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,492 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,492 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:11:12,493 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:11:12,493 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 09:11:12,493 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 09:11:12,493 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 09:11:12,493 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 09:11:12,493 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 09:11:12,493 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@35d57cb0} 09:11:12,493 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:12,493 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,494 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-1711641722] 09:11:12,494 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 09:11:12,495 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 09:11:12,496 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 09:11:12,496 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-1711641722] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,496 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-1180914280] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,496 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-1180914280 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,497 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,497 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,497 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-1711641722] 09:11:12,497 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,497 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-667910498] 09:11:12,498 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 09:11:12,498 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 09:11:12,499 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 09:11:12,500 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-667910498] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,500 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,500 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1574126247 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,500 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,500 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-667910498] 09:11:12,500 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1574126247] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,505 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,506 AM [cluster-test-shard-dispatcher-14] [INFO] Recovery - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed in in 6.481 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,506 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,506 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,506 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 09:11:12,506 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 09:11:12,507 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:11:12,507 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 09:11:12,507 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,507 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed in in 10.60 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,507 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,507 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,507 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 09:11:12,507 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 09:11:12,507 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 09:11:12,507 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:11:12,507 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 09:11:12,510 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 09:11:12,512 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:11:12,524 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-8228599015479651317] 09:11:12,524 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.4.0] ... 09:11:12,526 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:11:12,526 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 09:11:12,528 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:11:12,528 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:11:12,538 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,541 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,543 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 09:11:12,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1437404741] 09:11:12,560 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1437404741] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,560 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 09:11:12,561 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,563 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,564 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 09:11:12,564 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 09:11:12,564 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 09:11:12,564 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,565 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,565 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@6a3c927c} 09:11:12,565 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:12,565 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,569 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 09:11:12,570 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,570 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#1881593696] 09:11:12,570 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#1881593696] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,570 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 09:11:12,571 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 09:11:12,571 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#989432813 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,571 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#989432813] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,572 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 09:11:12,575 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard#-116890316] 09:11:12,578 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,579 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed in in 6.674 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,579 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard#-116890316] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,579 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 09:11:12,580 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 09:11:12,580 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-390268954] 09:11:12,580 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 09:11:12,581 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 09:11:12,581 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 09:11:12,581 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,581 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 09:11:12,581 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,581 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 09:11:12,581 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 09:11:12,583 AM [cluster-test-shard-dispatcher-20] [INFO] Recovery - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,583 AM [cluster-test-shard-dispatcher-20] [INFO] Recovery - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed in in 2.275 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,583 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,583 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,583 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@14b72e6f} 09:11:12,587 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 09:11:12,587 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is initialized 09:11:12,589 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,589 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,590 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1357289774 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,590 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:11:12,590 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 09:11:12,590 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:11:12,590 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 09:11:12,590 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 09:11:12,590 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1357289774] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,590 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-390268954] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,592 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:12,592 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,592 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#1743493784] 09:11:12,592 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#1743493784] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,592 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 09:11:12,594 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#1846223570 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,594 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 09:11:12,594 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#1846223570] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,594 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,595 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#1534331742] 09:11:12,595 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 09:11:12,595 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-868062500 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,596 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 09:11:12,596 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 09:11:12,597 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 09:11:12,599 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,600 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed in in 3.676 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,600 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,600 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,600 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 09:11:12,600 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 09:11:12,600 AM [cluster-test-shard-dispatcher-26] [INFO] Recovery - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,601 AM [cluster-test-shard-dispatcher-26] [INFO] Recovery - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed in in 3.955 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,601 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 09:11:12,601 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 09:11:12,601 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,601 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,601 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:11:12,601 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:11:12,601 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 09:11:12,601 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 09:11:12,601 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 09:11:12,601 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 09:11:12,602 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#1534331742] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,603 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-868062500] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,603 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 09:11:12,604 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 09:11:12,617 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:11:12,619 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,623 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,624 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:11:12,632 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [3515584260228515029] 09:11:12,633 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.4.0] ... 09:11:12,635 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:11:12,636 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 09:11:12,636 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 09:11:12,636 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 09:11:12,646 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,647 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,649 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 09:11:12,650 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#861626013] 09:11:12,651 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 09:11:12,651 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 09:11:12,651 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 09:11:12,651 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,651 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,653 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#861626013] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,656 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@7693d6e4} 09:11:12,657 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 09:11:12,657 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,659 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:11:12,659 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:12,659 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,660 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,660 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-884727636] 09:11:12,661 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 09:11:12,661 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 09:11:12,661 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 09:11:12,662 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 09:11:12,662 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-884727636] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,663 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1859261917] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,663 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1859261917 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,663 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#1163777529] 09:11:12,664 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#1163777529] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,664 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard#-215705217] 09:11:12,664 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,664 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard#-215705217] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,664 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 09:11:12,665 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed in in 2.903 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,665 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 09:11:12,665 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1053026364 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,665 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#1053026364] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,665 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 09:11:12,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 09:11:12,666 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 09:11:12,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,666 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 09:11:12,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,666 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:11:12,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007f52cc762520@4ae065f2} 09:11:12,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:11:12,666 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,667 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 09:11:12,668 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,668 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,668 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 09:11:12,668 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:11:12,668 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 09:11:12,668 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#-926752929] 09:11:12,668 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 09:11:12,669 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#97934866 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,669 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 09:11:12,670 AM [cluster-test-shard-dispatcher-17] [INFO] Recovery - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,670 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 09:11:12,671 AM [cluster-test-shard-dispatcher-17] [INFO] Recovery - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed in in 3.602 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,671 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,671 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,671 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:11:12,672 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 09:11:12,674 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,675 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#769595875] 09:11:12,675 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 09:11:12,675 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 09:11:12,676 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 09:11:12,676 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1415744099 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,677 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 09:11:12,677 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is initialized 09:11:12,677 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 09:11:12,677 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 09:11:12,678 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 09:11:12,678 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,679 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 09:11:12,680 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed in in 3.212 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,680 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 09:11:12,680 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 09:11:12,680 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,680 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,680 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:11:12,680 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 09:11:12,680 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 09:11:12,681 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed in in 8.381 ms: last log index = -1, last log term = -1, napshot index = -1, snapshot term = -1, journal size = 0 09:11:12,681 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 09:11:12,681 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 09:11:12,681 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,681 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,681 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:11:12,682 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 09:11:12,682 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#-926752929] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,682 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#97934866] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,682 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#769595875] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,682 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1415744099] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 09:11:12,682 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 09:11:12,682 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 09:11:12,686 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 09:11:12,686 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:12,686 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,686 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,686 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,686 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,686 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:12,686 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:12,686 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 09:11:12,687 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,687 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,687 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,687 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,687 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:12,687 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:12,842 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 09:11:12,842 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:11:12,842 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,842 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,843 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 09:11:12,843 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Follower to Candidate 09:11:12,873 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 09:11:12,873 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:11:12,873 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,873 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,873 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 09:11:12,873 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 09:11:12,893 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 09:11:12,893 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 09:11:12,893 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:11:12,893 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,893 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,893 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 09:11:12,893 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:11:12,893 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 09:11:12,893 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 09:11:12,893 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 09:11:12,893 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 09:11:12,893 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Follower to Candidate 09:11:13,166 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 09:11:13,224 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 09:11:13,256 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 09:11:13,256 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 09:11:13,481 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:13,482 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:11:13,482 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:13,482 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 09:11:13,482 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 09:11:13,482 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:11:13,482 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,482 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,490 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,491 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,491 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,543 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 09:11:13,545 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:13,545 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:13,553 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:11:13,554 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 09:11:13,554 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 09:11:13,554 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 09:11:13,554 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3dc87558 09:11:13,555 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Candidate to Leader 09:11:13,555 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 09:11:13,555 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@65730ad2 09:11:13,557 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@9f308a2 09:11:13,557 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 09:11:13,562 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:13,562 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 09:11:13,562 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,562 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,562 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,562 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,562 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,562 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,562 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,562 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:11:13,563 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,563 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,563 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,563 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,563 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,563 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,563 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,563 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 09:11:13,563 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,563 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,563 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,563 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,563 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,563 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,582 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 09:11:13,584 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:13,584 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:13,592 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:11:13,592 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 09:11:13,593 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 09:11:13,593 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 09:11:13,593 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@330f8c91 09:11:13,593 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 09:11:13,594 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 09:11:13,594 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5d277721 09:11:13,602 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3b4ce891 09:11:13,602 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 09:11:13,604 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 09:11:13,605 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:13,606 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:13,612 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:11:13,612 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 09:11:13,612 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@cce36b7 09:11:13,612 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 09:11:13,612 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 09:11:13,612 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Candidate to Leader 09:11:13,612 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 09:11:13,612 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 09:11:13,612 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:11:13,613 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 09:11:13,613 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@21bc1ba5 09:11:13,613 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 09:11:13,613 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@369a64bb 09:11:13,613 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 09:11:13,613 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 09:11:13,613 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 09:11:13,613 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1633215497] 09:11:13,613 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:11:13,613 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1116485252] 09:11:13,614 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1116485252] 09:11:13,614 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1437404741] 09:11:13,614 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:11:13,614 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#1881593696] 09:11:13,614 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#1881593696] 09:11:13,614 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#861626013] 09:11:13,614 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:11:13,614 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-884727636] 09:11:13,614 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-884727636] 09:11:13,615 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard ShardName{value=cars}: {member-2=false, member-3=false} 09:11:13,615 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 09:11:13,615 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:11:13,616 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,617 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]} has been successfully replicated to a majority of followers 09:11:13,618 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatusForShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,619 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 09:11:13,619 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] ClusterAdminRpcService - Successfully changed member voting states for shard ShardName{value=cars} 09:11:13,620 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#1633215497] 09:11:13,620 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 09:11:13,621 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1116485252] 09:11:13,621 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#-1116485252] 09:11:13,621 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1437404741] 09:11:13,621 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 09:11:13,621 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#1881593696] 09:11:13,621 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#1881593696] 09:11:13,624 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 09:11:13,624 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 09:11:13,624 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 09:11:13,625 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#861626013] 09:11:13,625 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 09:11:13,625 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-884727636] 09:11:13,625 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-884727636] 09:11:13,625 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:11:13,625 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:13,625 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,625 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,625 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 09:11:13,626 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 09:11:13,626 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 09:11:13,626 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 09:11:13,626 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 09:11:13,626 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 09:11:13,626 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 09:11:13,626 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 09:11:13,627 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 09:11:13,627 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,627 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,627 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 09:11:13,627 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 09:11:13,628 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 09:11:13,631 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 09:11:13,632 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 09:11:13,632 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 09:11:13,632 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 09:11:13,633 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:11:13,633 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 09:11:13,633 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 09:11:13,633 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 09:11:13,633 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:11:13,633 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@438a3111 09:11:13,634 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 09:11:13,634 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 09:11:13,634 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 09:11:13,634 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3fcb89f3 09:11:13,634 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 09:11:13,634 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:11:13,637 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:11:13,637 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:11:13,637 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:11:13,637 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:11:13,641 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:11:13,656 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:11:13,656 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:13,656 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 09:11:13,656 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 09:11:13,657 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 09:11:13,657 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 09:11:13,657 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 09:11:13,657 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 09:11:13,658 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 09:11:13,658 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 09:11:13,658 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 09:11:13,659 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:13,659 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:13,659 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 09:11:13,659 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 09:11:13,659 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 09:11:13,659 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 09:11:13,659 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 09:11:13,659 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 09:11:13,659 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 09:11:13,661 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:11:13,661 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:11:14,500 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 09:11:14,651 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [3515584260228515029] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:11:14,680 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:11:14,680 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:11:14,680 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-8228599015479651317] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:11:14,682 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:11:14.684] [cluster-test-pekko.actor.internal-dispatcher-5] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:11:14,688 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 09:11:14,689 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 09:11:14,689 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 09:11:14,689 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 09:11:14,690 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@c280500 09:11:14,690 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@747ca616 09:11:14,691 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@c280500 09:11:14,691 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 09:11:14,691 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@747ca616 09:11:14,691 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 09:11:14,691 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@121d79e5 09:11:14,692 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6cb56ae0 09:11:15,150 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer aborted 09:11:15,151 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 461.2 ms 09:11:15,151 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In unpauseLeader 09:11:15,151 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 09:11:15,152 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 09:11:15,210 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer aborted 09:11:15,210 AM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 520.9 ms 09:11:15,211 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In unpauseLeader 09:11:15,211 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 09:11:15,212 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 09:11:15,212 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 09:11:15,212 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 09:11:15,212 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 09:11:15,213 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 09:11:15,213 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 09:11:15,213 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 09:11:15,213 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@541a7a32 09:11:15,213 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@541a7a32 09:11:15,213 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 09:11:15,213 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7ea5ccde 09:11:15,214 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6f78890a 09:11:15,214 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@352acdfd 09:11:15,214 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@352acdfd 09:11:15,214 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 09:11:15,550 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 09:11:15,550 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 336.6 ms 09:11:15,550 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 09:11:15,551 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 09:11:15,552 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 09:11:15,830 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 09:11:15,830 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] ShardManager - shard-manager-oper-member-1: An error occurred attempting to shut down the shards java.util.concurrent.CompletionException: org.apache.pekko.pattern.AskTimeoutException: Ask timed out on [Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-667910498]] after [600 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347) at java.base/java.util.concurrent.CompletableFuture.biApply(CompletableFuture.java:1301) at java.base/java.util.concurrent.CompletableFuture$BiApply.tryFire(CompletableFuture.java:1280) at java.base/java.util.concurrent.CompletableFuture$CoCompletion.tryFire(CompletableFuture.java:1219) at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194) at scala.concurrent.impl.FutureConvertersImpl$CF.apply(FutureConvertersImpl.scala:26) at scala.concurrent.impl.FutureConvertersImpl$CF.apply(FutureConvertersImpl.scala:23) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:517) at scala.concurrent.ExecutionContext$parasitic$.execute(ExecutionContext.scala:222) at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:462) at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:371) at scala.concurrent.impl.Promise$DefaultPromise.tryComplete0(Promise.scala:295) at scala.concurrent.impl.Promise$Transformation.handleFailure(Promise.scala:477) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:539) at scala.concurrent.ExecutionContext$parasitic$.execute(ExecutionContext.scala:222) at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:462) at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:368) at scala.concurrent.impl.Promise$DefaultPromise.tryComplete0(Promise.scala:295) at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:288) at org.apache.pekko.pattern.PromiseActorRef$.$anonfun$1(AskSupport.scala:733) at org.apache.pekko.pattern.PromiseActorRef$.$anonfun$adapted$1(AskSupport.scala:738) at scala.Function0.apply$mcV$sp(Function0.scala:42) at org.apache.pekko.actor.Scheduler$$anon$7.run(Scheduler.scala:468) at scala.concurrent.ExecutionContext$parasitic$.execute(ExecutionContext.scala:222) at org.apache.pekko.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(LightArrayRevolverScheduler.scala:387) at org.apache.pekko.actor.LightArrayRevolverScheduler$$anon$1.executeBucket$1(LightArrayRevolverScheduler.scala:333) at org.apache.pekko.actor.LightArrayRevolverScheduler$$anon$1.nextTick(LightArrayRevolverScheduler.scala:337) at org.apache.pekko.actor.LightArrayRevolverScheduler$$anon$1.run(LightArrayRevolverScheduler.scala:289) at java.base/java.lang.Thread.run(Thread.java:1583) Caused by: org.apache.pekko.pattern.AskTimeoutException: Ask timed out on [Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#-667910498]] after [600 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 09:11:15,830 AM [cluster-test-shard-dispatcher-13] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 09:11:15,831 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 618.1 ms 09:11:15,832 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 09:11:15,833 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 09:11:15,834 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 09:11:15,835 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 09:11:15,837 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:11:15,837 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:11:15,840 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/18/2026 09:11:15.841] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config