08:53:00,736 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:53:01,303 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-4720688132461503696] 08:53:01,324 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:53:01,501 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:53:01,502 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:53:01,556 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:01,557 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:01,921 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:53:01,922 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:53:01,926 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:01,937 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:01,941 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:53:02,918 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testBackupDatastore 08:53:03,32 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-238684447] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:03,51 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:53:03,52 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:53:03,105 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:53:03,111 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testBackupDatastore 08:53:03,119 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 08:53:03,166 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testBackupDatastore 08:53:03,227 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:03,228 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 08:53:03,228 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 08:53:03,229 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@7cfb5bf9} 08:53:03,247 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} started 08:53:03,253 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:03,253 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testBackupDatastore 08:53:03,255 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 08:53:03,255 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore 08:53:03,256 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore on shard member-1-shard-cars-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#953506329] 08:53:03,256 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testBackupDatastore 08:53:03,262 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 08:53:03,262 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore to pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore 08:53:03,263 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore on shard member-1-shard-people-config_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-1087797331] 08:53:03,265 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#953506329] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:03,265 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-1087797331] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:03,299 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Shard created, persistent : true 08:53:03,299 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Shard created, persistent : true 08:53:03,302 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-238684447] 08:53:03,303 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:03,303 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:53:03,354 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-238684447] 08:53:03,355 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:03,355 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:53:03,406 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-238684447] 08:53:03,406 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:03,407 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:53:03,435 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#-81621701] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:03,437 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testBackupDatastore: Starting recovery with journal batch size 1 08:53:03,437 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore-notifier#-81621701 created and ready for shard:member-1-shard-people-config_testBackupDatastore 08:53:03,442 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#1275788184 created and ready for shard:member-1-shard-cars-config_testBackupDatastore 08:53:03,442 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore-notifier#1275788184] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:03,443 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-cars-config_testBackupDatastore: Starting recovery with journal batch size 1 08:53:03,458 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-238684447] 08:53:03,459 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:03,459 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:53:03,509 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-238684447] 08:53:03,512 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:03,512 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:53:03,563 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-238684447] 08:53:03,563 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:03,564 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:53:03,614 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-238684447] 08:53:03,615 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:03,615 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:53:03,622 AM [cluster-test-shard-dispatcher-11] [INFO] FileMapper - Using io.netty.util.internal.PlatformDependent for ByteBuffer cleanup 08:53:03,630 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testBackupDatastore: journal open: applyTo=0 08:53:03,630 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testBackupDatastore: journal open: applyTo=0 08:53:03,656 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:03,656 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:03,665 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-238684447] 08:53:03,666 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:03,666 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:53:03,673 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 08:53:03,675 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-config_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 08:53:03,685 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:53:03,685 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testBackupDatastore is initialized 08:53:03,694 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:53:03,695 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:03,697 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:53:03,697 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:03,698 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:53:03,698 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testBackupDatastore is initialized 08:53:03,700 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from null to Follower 08:53:03,700 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from null to Follower 08:53:03,716 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-238684447] 08:53:03,717 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:03,717 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#953506329] 08:53:03,741 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-cars-config_testBackupDatastore (Candidate): Starting new election term 1 08:53:03,741 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:03,741 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:53:03,741 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testBackupDatastore (Candidate): Starting new election term 1 08:53:03,741 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:03,742 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:03,742 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Follower to Candidate 08:53:03,742 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:53:03,743 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:03,743 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Follower to Candidate 08:53:03,746 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:03,746 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-config_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:03,746 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 08:53:03,746 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 08:53:03,756 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: became leader with frontend state for [] 08:53:03,756 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testBackupDatastore: became leader with frontend state for [] 08:53:03,757 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , received role change from Candidate to Leader 08:53:03,757 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , received role change from Candidate to Leader 08:53:03,764 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 08:53:03,764 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-config_testBackupDatastore 08:53:03,765 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6bf8e3ec 08:53:03,765 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@39082c1f 08:53:03,766 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testBackupDatastore from Candidate to Leader 08:53:03,766 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testBackupDatastore from Candidate to Leader 08:53:03,766 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testBackupDatastore is ready 08:53:03,816 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-238684447] 08:53:03,817 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:53:03,817 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-people-config_testBackupDatastore#-1087797331] 08:53:03,820 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:03,830 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:03,836 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testBackupDatastore 08:53:03,839 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:53:03,840 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:53:03,840 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#751342720] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:03,845 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:53:03,845 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testBackupDatastore 08:53:03,846 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testBackupDatastore 08:53:03,846 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:03,846 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 08:53:03,846 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 08:53:03,851 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testBackupDatastore: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 08:53:03,853 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@28a49d37} 08:53:03,853 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} started 08:53:03,854 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:03,854 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testBackupDatastore 08:53:03,856 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 08:53:03,856 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore 08:53:03,856 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore on shard member-1-shard-cars-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-980747155] 08:53:03,856 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testBackupDatastore 08:53:03,857 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Shard created, persistent : true 08:53:03,858 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-cars-oper_testBackupDatastore: Starting recovery with journal batch size 1 08:53:03,858 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Shard created, persistent : true 08:53:03,859 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-oper_testBackupDatastore: Starting recovery with journal batch size 1 08:53:03,860 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testBackupDatastore: journal open: applyTo=0 08:53:03,861 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 08:53:03,864 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testBackupDatastore with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore to pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore 08:53:03,865 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testBackupDatastore on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore on shard member-1-shard-people-oper_testBackupDatastore whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-1514311780] 08:53:03,861 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-980747155] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:03,865 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-1514311780] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:03,865 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#-1922696034] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:03,861 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testBackupDatastore: journal open: applyTo=0 08:53:03,867 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore-notifier#-1922696034 created and ready for shard:member-1-shard-cars-oper_testBackupDatastore 08:53:03,867 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:03,867 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-oper_testBackupDatastore: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:03,868 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#1291496233] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:03,867 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore-notifier#1291496233 created and ready for shard:member-1-shard-people-oper_testBackupDatastore 08:53:03,881 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 08:53:03,882 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:53:03,882 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:03,883 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from null to Follower 08:53:03,883 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:53:03,883 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testBackupDatastore is initialized 08:53:03,884 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-oper_testBackupDatastore: Local TermInfo store seeded with TermInfo{term=0} 08:53:03,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:53:03,885 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:53:03,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:03,885 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testBackupDatastore is initialized 08:53:03,885 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from null to Follower 08:53:03,904 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#751342720] 08:53:03,905 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:53:03,905 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-cars-oper_testBackupDatastore#-980747155] 08:53:03,926 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testBackupDatastore (Candidate): Starting new election term 1 08:53:03,926 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:03,926 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:53:03,926 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:03,927 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Follower to Candidate 08:53:03,927 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:03,927 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 08:53:03,927 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: became leader with frontend state for [] 08:53:03,927 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testBackupDatastore (Candidate): Starting new election term 1 08:53:03,928 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testBackupDatastore (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:03,928 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: false, hasLeader : false 08:53:03,928 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:03,929 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Follower to Candidate 08:53:03,929 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , received role change from Candidate to Leader 08:53:03,929 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 08:53:03,929 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testBackupDatastore (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:03,930 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testBackupDatastore: onLeadershipChange, isLeader: true, hasLeader : true 08:53:03,930 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4b9d0e51 08:53:03,930 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: became leader with frontend state for [] 08:53:03,930 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testBackupDatastore from Candidate to Leader 08:53:03,931 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , received role change from Candidate to Leader 08:53:03,931 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testBackupDatastore , registered listener pekko://cluster-test/user/shardmanager-oper_testBackupDatastore 08:53:03,932 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@a10c581 08:53:03,932 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testBackupDatastore from Candidate to Leader 08:53:03,932 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testBackupDatastore is ready 08:53:03,981 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore#751342720] 08:53:03,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:53:03,982 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testBackupDatastore/member-1-shard-people-oper_testBackupDatastore#-1514311780] 08:53:04,0 AM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 08:53:04,2 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 08:53:04,5 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 08:53:04,17 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 08:53:04,18 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 08:53:04,25 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@32d7a9cc, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 08:53:04,33 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5f390833, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 08:53:04,34 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 08:53:04,28 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-cars-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5ddc9ed6, electionTerm=1, electionVotedFor=member-1-shard-cars-config_testBackupDatastore, ServerConfigPayload=null]] 08:53:04,34 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@484b7c3d, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 08:53:04,34 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: All shard snapshots received 08:53:04,79 AM [main] [INFO] ClusterAdminRpcService - Successfully backed up datastore to file target/testBackupDatastore 08:53:04,107 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore#-238684447] 08:53:04,109 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:04,110 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testBackupDatastore/member-1-shard-cars-config_testBackupDatastore#953506329] 08:53:04,145 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testBackupDatastore: Stopping Shard 08:53:04,148 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testBackupDatastore: journal closed 08:53:04,154 AM [main] [DEBUG] ClusterAdminRpcService - backupDatastore: BackupDatastoreInput{filePath=target/testBackupDatastore} 08:53:04,155 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onGetSnapshot 08:53:04,155 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: onGetSnapshot 08:53:04,157 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Expecting 2 shard snapshot replies 08:53:04,157 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-cars-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1e640c6a, electionTerm=1, electionVotedFor=member-1-shard-cars-oper_testBackupDatastore, ServerConfigPayload=null]] 08:53:04,158 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: Received GetSnapshotReply[id=member-1-shard-people-oper_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@29e134e5, electionTerm=1, electionVotedFor=member-1-shard-people-oper_testBackupDatastore, ServerConfigPayload=null]] 08:53:04,158 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-oper-member-1: All shard snapshots received 08:53:04,158 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Expecting 2 shard snapshot replies 08:53:04,161 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Received GetSnapshotReply[id=member-1-shard-people-config_testBackupDatastore, snapshot=Snapshot [lastIndex=-1, lastTerm=-1, lastAppliedIndex=-1, lastAppliedTerm=-1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6a93d4d5, electionTerm=1, electionVotedFor=member-1-shard-people-config_testBackupDatastore, ServerConfigPayload=null]] 08:53:04,374 AM [cluster-test-pekko.actor.default-dispatcher-20] [WARN] ShardManagerGetSnapshotReplyActor - shard-manager-config-member-1: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond 08:53:04,377 AM [cluster-test-pekko.actor.default-dispatcher-20] [ERROR] ClusterAdminRpcService - Failed to back up datastore to file target/testBackupDatastore java.util.concurrent.TimeoutException: Timed out after 200 ms while waiting for snapshot replies from 2 shard(s). 1 shard(s) [cars] did not respond. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManagerGetSnapshotReplyActor.onReceive(ShardManagerGetSnapshotReplyActor.java:66) at org.apache.pekko.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:341) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.actor.AbstractActor.aroundReceive(AbstractActor.scala:229) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 08:53:04,388 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testBackupDatastore, generation=0} 08:53:04,391 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testBackupDatastore 08:53:04,392 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testBackupDatastore 08:53:04,392 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:53:04,395 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@5a4b1a8c 08:53:04,396 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-config_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@5a4b1a8c 08:53:04,398 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testBackupDatastore: Stopping Shard 08:53:04,398 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-config_testBackupDatastore: journal closed 08:53:04,399 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:53:04,399 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:53:04,402 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testBackupDatastore, generation=0} 08:53:04,402 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testBackupDatastore 08:53:04,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@6951ae2f 08:53:04,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@6951ae2f 08:53:04,403 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testBackupDatastore 08:53:04,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testBackupDatastore: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@1d7e89ec 08:53:04,403 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-oper_testBackupDatastore: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@1d7e89ec 08:53:04,404 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:53:04,404 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testBackupDatastore: Stopping Shard 08:53:04,404 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testBackupDatastore: Stopping Shard 08:53:04,404 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testBackupDatastore: journal closed 08:53:04,405 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testBackupDatastore: journal closed 08:53:04,405 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:53:04,405 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:53:04,410 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:04,411 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:04,433 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:53:04,491 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:53:04,522 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [1648000754343237505] 08:53:04,522 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:53:04,527 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:53:04,528 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:53:04,529 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:04,530 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:04,563 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:04,564 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:53:04,564 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:53:04,564 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:53:04,570 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:04,573 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 08:53:04,581 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#530913683] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:04,582 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:53:04,582 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:53:04,592 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:53:04,592 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddReplicasForAllShards 08:53:04,593 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testAddReplicasForAllShards 08:53:04,594 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:04,594 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 08:53:04,595 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 08:53:04,643 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 08:53:04,645 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@1511311d} 08:53:04,646 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 08:53:04,648 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:04,648 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddReplicasForAllShards 08:53:04,649 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 08:53:04,650 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 08:53:04,650 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards on shard member-1-shard-cars-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-908843055] 08:53:04,650 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testAddReplicasForAllShards 08:53:04,651 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 08:53:04,652 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 08:53:04,652 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards on shard member-1-shard-people-config_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#820784242] 08:53:04,653 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Shard created, persistent : true 08:53:04,653 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-908843055] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:04,657 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#-1100466234 created and ready for shard:member-1-shard-cars-config_testAddReplicasForAllShards 08:53:04,657 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards-notifier#-1100466234] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:04,658 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:53:04,661 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddReplicasForAllShards: journal open: applyTo=0 08:53:04,662 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Shard created, persistent : true 08:53:04,663 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-people-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:53:04,663 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#-1889800096 created and ready for shard:member-1-shard-people-config_testAddReplicasForAllShards 08:53:04,663 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-config_testAddReplicasForAllShards: journal open: applyTo=0 08:53:04,664 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#820784242] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:04,665 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:04,666 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:04,667 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards-notifier#-1889800096] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:04,697 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#530913683] 08:53:04,698 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:04,698 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Returning NotInitializedException for shard cars 08:53:04,723 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:53:04,724 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:04,724 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:04,725 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 08:53:04,726 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-people-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:53:04,727 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:04,727 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:04,727 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 08:53:04,726 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:53:04,728 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddReplicasForAllShards is initialized 08:53:04,729 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:53:04,729 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testAddReplicasForAllShards is initialized 08:53:04,730 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 08:53:04,730 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:04,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:04,730 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:04,730 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Follower to Candidate 08:53:04,730 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:04,731 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:53:04,731 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: became leader with frontend state for [] 08:53:04,731 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Candidate to Leader 08:53:04,731 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 08:53:04,731 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@73f25c81 08:53:04,731 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Candidate to Leader 08:53:04,748 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-people-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 08:53:04,748 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:04,748 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:04,748 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:04,748 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Follower to Candidate 08:53:04,748 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#530913683] 08:53:04,749 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:04,749 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:53:04,749 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: became leader with frontend state for [] 08:53:04,750 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Candidate to Leader 08:53:04,750 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:04,750 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 08:53:04,750 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-908843055] 08:53:04,751 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@8969ceb 08:53:04,751 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Candidate to Leader 08:53:04,751 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:53:04,801 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#530913683] 08:53:04,801 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:53:04,801 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#820784242] 08:53:04,802 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:04,807 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:04,812 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 08:53:04,814 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-51790751] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:04,817 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:53:04,818 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:53:04,819 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:53:04,819 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddReplicasForAllShards 08:53:04,819 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testAddReplicasForAllShards 08:53:04,819 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:04,820 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 08:53:04,820 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 08:53:04,845 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 08:53:04,847 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@61b0fbfb} 08:53:04,847 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 08:53:04,848 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:04,848 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddReplicasForAllShards 08:53:04,851 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Shard created, persistent : true 08:53:04,851 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-1641144965] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:04,852 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#1319106367 created and ready for shard:member-1-shard-cars-oper_testAddReplicasForAllShards 08:53:04,852 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-cars-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:53:04,852 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards-notifier#1319106367] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:04,854 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 08:53:04,854 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 08:53:04,854 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards on shard member-1-shard-cars-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-1641144965] 08:53:04,854 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testAddReplicasForAllShards 08:53:04,855 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddReplicasForAllShards: journal open: applyTo=0 08:53:04,856 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Shard created, persistent : true 08:53:04,856 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:53:04,857 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testAddReplicasForAllShards: journal open: applyTo=0 08:53:04,858 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#-1107113282 created and ready for shard:member-1-shard-people-oper_testAddReplicasForAllShards 08:53:04,858 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#19658949] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:04,859 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:04,860 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:04,860 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 08:53:04,860 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 08:53:04,860 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testAddReplicasForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards on shard member-1-shard-people-oper_testAddReplicasForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#19658949] 08:53:04,861 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards-notifier#-1107113282] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:04,879 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:53:04,880 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:04,880 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:04,881 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 08:53:04,882 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:53:04,882 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testAddReplicasForAllShards is initialized 08:53:04,889 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:53:04,890 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:04,891 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:04,892 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:53:04,892 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddReplicasForAllShards is initialized 08:53:04,892 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 08:53:04,897 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 08:53:04,897 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:04,897 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:04,897 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:04,897 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:04,897 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:53:04,898 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: became leader with frontend state for [] 08:53:04,898 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate): Starting new election term 1 08:53:04,898 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:04,898 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:04,898 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-51790751] 08:53:04,899 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:53:04,899 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-1641144965] 08:53:04,898 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:04,899 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 08:53:04,899 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:04,899 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 08:53:04,899 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:53:04,900 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: became leader with frontend state for [] 08:53:04,900 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 08:53:04,900 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Follower to Candidate 08:53:04,900 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Candidate to Leader 08:53:04,900 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e947b57 08:53:04,900 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 08:53:04,900 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Candidate to Leader 08:53:04,900 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3cd4d330 08:53:04,901 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Candidate to Leader 08:53:04,901 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 08:53:04,950 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-51790751] 08:53:04,951 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:53:04,951 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#19658949] 08:53:04,957 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@5d9a3164] 08:53:04,963 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: true 08:53:04,966 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Shard created, persistent : true 08:53:04,967 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-pets-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:53:04,968 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testAddReplicasForAllShards: journal open: applyTo=0 08:53:04,969 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards-notifier#-974157266 created and ready for shard:member-1-shard-pets-config_testAddReplicasForAllShards 08:53:04,970 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:04,989 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-pets-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:53:04,990 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:04,990 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:04,990 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 08:53:04,992 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 08:53:04,992 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testAddReplicasForAllShards is initialized 08:53:04,999 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate): Starting new election term 1 08:53:04,999 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:04,999 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:04,999 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:05,0 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Follower to Candidate 08:53:05,0 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:05,0 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:53:05,0 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: became leader with frontend state for [] 08:53:05,0 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Candidate to Leader 08:53:05,0 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 08:53:05,1 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@d2d479b 08:53:05,1 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Candidate to Leader 08:53:05,1 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:53:05,22 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#530913683] 08:53:05,22 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 08:53:05,23 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#1867142625] 08:53:05,39 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:53:05,63 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [7310673253511464277] 08:53:05,63 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 08:53:05,71 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:05,71 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:53:05,73 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:05,73 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:05,111 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:05,114 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:05,117 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddReplicasForAllShards 08:53:05,145 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1465586145] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:05,150 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:53:05,151 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:53:05,160 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:53:05,187 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 08:53:05,191 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@789c881c} 08:53:05,191 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} started 08:53:05,192 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:05,195 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:05,195 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:05,198 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddReplicasForAllShards 08:53:05,202 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:53:05,202 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:53:05,202 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1011249537] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:05,203 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:53:05,210 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddReplicasForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 08:53:05,212 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@765b697f} 08:53:05,212 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} started 08:53:05,212 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:05,540 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:53:05,549 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:53:05,549 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:05,549 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards 08:53:05,549 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards 08:53:05,549 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards 08:53:05,549 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards 08:53:05,549 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 08:53:05,550 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:05,550 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards 08:53:05,550 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-pets-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards 08:53:05,554 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards 08:53:05,554 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards 08:53:05,554 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards 08:53:05,554 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testAddReplicasForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards to pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards 08:53:05,554 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:53:05,628 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:53:05,637 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:05,637 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 08:53:05,637 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:05,637 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:53:05,649 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:05,649 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddReplicasForAllShards is ready 08:53:05,649 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:05,649 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:53:05,738 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@7264e36c] 08:53:05,742 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 08:53:05,744 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=no-leader-ns, moduleName=no-leader-module, shardName=no-leader, shardMemberNames=[MemberName{name=member-1}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@5331b96a] 08:53:05,745 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Shard created, persistent : true 08:53:05,745 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#1562675100] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:05,746 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-2 doCreateShard: shardId: member-2-shard-no-leader-oper_testAddReplicasForAllShards, memberNames: [MemberName{name=member-1}], peerAddresses: {}, isActiveMember: false 08:53:05,746 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-pets-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:53:05,746 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#1865504652 created and ready for shard:member-2-shard-pets-config_testAddReplicasForAllShards 08:53:05,747 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-notifier#1865504652] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:05,748 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards#-1145676378] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:05,748 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Shard created, persistent : true 08:53:05,749 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#-1858001784 created and ready for shard:member-2-shard-no-leader-oper_testAddReplicasForAllShards 08:53:05,749 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards/member-2-shard-no-leader-oper_testAddReplicasForAllShards-notifier#-1858001784] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:05,749 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:53:05,751 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-no-leader-oper_testAddReplicasForAllShards: journal open: applyTo=0 08:53:05,754 AM [main] [INFO] ClusterAdminRpcService - Adding replicas for all shards 08:53:05,755 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testAddReplicasForAllShards: journal open: applyTo=0 08:53:05,755 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testAddReplicasForAllShards 08:53:05,755 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 08:53:05,756 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, no-leader] for data store oper_testAddReplicasForAllShards 08:53:05,756 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 08:53:05,759 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 08:53:05,759 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=no-leader] 08:53:05,761 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-2-shard-pets-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:05,761 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:05,762 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=people] 08:53:05,762 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=pets] 08:53:05,763 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:05,764 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 08:53:05,759 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:05,765 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 08:53:05,767 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 08:53:05,767 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 08:53:05,769 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 08:53:05,769 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=no-leader, waitUntilReady=true] 08:53:05,769 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for people forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 08:53:05,769 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for no-leader forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards] 08:53:05,770 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 08:53:05,770 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for pets forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards] 08:53:05,775 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:53:05,777 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:53:05,777 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-908843055, primaryVersion=13] 08:53:05,777 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-1641144965, primaryVersion=13] 08:53:05,777 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 08:53:05,777 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=people, waitUntilReady=true] 08:53:05,777 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#19658949, primaryVersion=13] 08:53:05,777 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#820784242, primaryVersion=13] 08:53:05,778 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=no-leader, waitUntilReady=true] 08:53:05,778 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: No shard found for no-leader 08:53:05,778 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=pets, waitUntilReady=true] 08:53:05,778 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#1867142625, primaryVersion=13] 08:53:05,781 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-2-shard-pets-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:53:05,782 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:05,782 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:05,783 AM [cluster-test-shard-dispatcher-10] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 08:53:05,783 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 08:53:05,783 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testAddReplicasForAllShards is initialized 08:53:05,784 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , received role change from null to Follower 08:53:05,784 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 08:53:05,784 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testAddReplicasForAllShards from null to Follower 08:53:05,799 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:53:05,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:05,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:05,800 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [no-leader] 08:53:05,800 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-no-leader-oper_testAddReplicasForAllShards is initialized 08:53:05,800 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 08:53:05,800 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , received role change from null to Follower 08:53:05,800 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-no-leader-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 08:53:05,801 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-no-leader-oper_testAddReplicasForAllShards from null to Follower 08:53:05,802 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-908843055 for shard member-2-shard-cars-config_testAddReplicasForAllShards 08:53:05,804 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Shard created, persistent : true 08:53:05,805 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-2-shard-cars-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:53:05,806 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddReplicasForAllShards: journal open: applyTo=0 08:53:05,806 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#848128064] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:05,807 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#433259108 created and ready for shard:member-2-shard-cars-config_testAddReplicasForAllShards 08:53:05,808 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-notifier#433259108] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:05,809 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-1641144965 for shard member-2-shard-cars-oper_testAddReplicasForAllShards 08:53:05,810 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Shard created, persistent : true 08:53:05,811 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-1515580869] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:05,812 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:05,812 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#-1097078047 created and ready for shard:member-2-shard-cars-oper_testAddReplicasForAllShards 08:53:05,812 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-notifier#-1097078047] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:05,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#19658949 for shard member-2-shard-people-oper_testAddReplicasForAllShards 08:53:05,818 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#820784242 for shard member-2-shard-people-config_testAddReplicasForAllShards 08:53:05,815 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddReplicasForAllShards 08:53:05,819 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Shard created, persistent : true 08:53:05,815 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddReplicasForAllShards 08:53:05,819 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Shard created, persistent : true 08:53:05,818 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-oper_testAddReplicasForAllShards 08:53:05,819 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#1867142625 for shard member-2-shard-pets-config_testAddReplicasForAllShards 08:53:05,820 AM [cluster-test-shard-dispatcher-16] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-people-config_testAddReplicasForAllShards 08:53:05,821 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-2-shard-people-config_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:53:05,825 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-2-shard-people-config_testAddReplicasForAllShards: journal open: applyTo=0 08:53:05,823 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-notifier#382095043 created and ready for shard:member-2-shard-people-config_testAddReplicasForAllShards 08:53:05,828 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-2-shard-people-config_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:05,822 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:53:05,822 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-pets-config_testAddReplicasForAllShards 08:53:05,829 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:53:05,829 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddReplicasForAllShards is initialized 08:53:05,829 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-people-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:53:05,829 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:05,829 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:05,829 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , received role change from null to Follower 08:53:05,829 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 08:53:05,830 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddReplicasForAllShards from null to Follower 08:53:05,830 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testAddReplicasForAllShards: journal open: applyTo=0 08:53:05,830 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-notifier#1072768534 created and ready for shard:member-2-shard-people-oper_testAddReplicasForAllShards 08:53:05,831 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: Failed to find leader for shard no-leader: received response: {} org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1036) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:229) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:225) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:224) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:280) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:281) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 08:53:05,831 AM [cluster-test-pekko.actor.default-dispatcher-11] [WARN] ClusterAdminRpcService - Failed to add replica for shard ShardName{value=no-leader}, type Operational org.opendaylight.controller.cluster.datastore.exceptions.PrimaryNotFoundException: No primary shard found for no-leader. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.findPrimary(ShardManager.java:1036) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:229) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:225) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:224) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:280) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:281) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 08:53:05,830 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Starting recovery with journal batch size 1 08:53:05,833 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddReplicasForAllShards: journal open: applyTo=0 08:53:05,834 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:05,832 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-oper_testAddReplicasForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:05,837 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 1 08:53:05,837 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-2-shard-people-config_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:53:05,838 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:53:05,838 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testAddReplicasForAllShards is initialized 08:53:05,838 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:05,838 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:05,838 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , received role change from null to Follower 08:53:05,838 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards 08:53:05,839 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:53:05,839 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:53:05,839 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:53:05,841 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:53:05,841 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:05,841 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:53:05,842 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:05,841 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testAddReplicasForAllShards from null to Follower 08:53:05,841 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:05,846 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:05,847 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddReplicasForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:53:05,847 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:53:05,847 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddReplicasForAllShards is initialized 08:53:05,847 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:05,847 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:05,847 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , received role change from null to Follower 08:53:05,848 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:53:05,848 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testAddReplicasForAllShards is initialized 08:53:05,848 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:53:05,848 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:05,848 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , received role change from null to Follower 08:53:05,848 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 08:53:05,849 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testAddReplicasForAllShards from null to Follower 08:53:05,849 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddReplicasForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards 08:53:05,849 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddReplicasForAllShards from null to Follower 08:53:05,850 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:53:05,850 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:05,851 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:53:05,853 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-people-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:53:05,853 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-pets-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:53:05,854 AM [cluster-test-shard-dispatcher-10] [INFO] Follower - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:53:05,854 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-people-oper_testAddReplicasForAllShards, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:53:05,879 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-cars-oper_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddReplicasForAllShards 08:53:05,880 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-people-oper_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-people-oper_testAddReplicasForAllShards 08:53:05,880 AM [cluster-test-shard-dispatcher-10] [INFO] Follower - member-2-shard-cars-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddReplicasForAllShards 08:53:05,881 AM [cluster-test-shard-dispatcher-31] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:05,881 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:05,881 AM [cluster-test-shard-dispatcher-10] [INFO] SnapshotManager - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:05,882 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-pets-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-pets-config_testAddReplicasForAllShards 08:53:05,883 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:05,883 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@53461ab7 08:53:05,883 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done false 08:53:05,883 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done false 08:53:05,884 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-1-shard-people-config_testAddReplicasForAllShards: snapshot is durable as of 2026-04-10T08:53:05.841357608Z 08:53:05,885 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-config_testAddReplicasForAllShards (Follower): Snapshot received from leader: member-1-shard-people-config_testAddReplicasForAllShards 08:53:05,885 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:05,886 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3eecf66d 08:53:05,886 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done false 08:53:05,887 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3596936d 08:53:05,887 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:53:05,889 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddReplicasForAllShards: snapshot is durable as of 2026-04-10T08:53:05.846069393Z 08:53:05,889 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddReplicasForAllShards: snapshot is durable as of 2026-04-10T08:53:05.841948680Z 08:53:05,890 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-pets-config_testAddReplicasForAllShards: snapshot is durable as of 2026-04-10T08:53:05.841358668Z 08:53:05,890 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-people-oper_testAddReplicasForAllShards: snapshot is durable as of 2026-04-10T08:53:05.850727525Z 08:53:05,892 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@29fd0a77 08:53:05,893 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2b2bc1d1 08:53:05,893 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done false 08:53:05,893 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done false 08:53:05,914 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-2-shard-cars-config_testAddReplicasForAllShards: snapshot is durable as of 2026-04-10T08:53:05.887668724Z 08:53:05,914 AM [cluster-test-shard-dispatcher-32] [INFO] SnapshotManager - member-2-shard-pets-config_testAddReplicasForAllShards: snapshot is durable as of 2026-04-10T08:53:05.883932189Z 08:53:05,914 AM [cluster-test-shard-dispatcher-29] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddReplicasForAllShards: Applying snapshot 08:53:05,914 AM [cluster-test-shard-dispatcher-32] [INFO] ShardSnapshotCohort - member-2-shard-pets-config_testAddReplicasForAllShards: Applying snapshot 08:53:05,915 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:05,915 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-pets-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:05,917 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddReplicasForAllShards: snapshot is durable as of 2026-04-10T08:53:05.883122133Z 08:53:05,918 AM [cluster-test-shard-dispatcher-28] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddReplicasForAllShards: Applying snapshot 08:53:05,918 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:05,919 AM [cluster-test-shard-dispatcher-27] [INFO] SnapshotManager - member-2-shard-people-config_testAddReplicasForAllShards: snapshot is durable as of 2026-04-10T08:53:05.886367968Z 08:53:05,919 AM [cluster-test-shard-dispatcher-27] [INFO] ShardSnapshotCohort - member-2-shard-people-config_testAddReplicasForAllShards: Applying snapshot 08:53:05,919 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-people-config_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:05,920 AM [cluster-test-shard-dispatcher-30] [INFO] SnapshotManager - member-2-shard-people-oper_testAddReplicasForAllShards: snapshot is durable as of 2026-04-10T08:53:05.890448280Z 08:53:05,921 AM [cluster-test-shard-dispatcher-30] [INFO] ShardSnapshotCohort - member-2-shard-people-oper_testAddReplicasForAllShards: Applying snapshot 08:53:05,921 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-people-oper_testAddReplicasForAllShards: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:05,943 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#-1670530426] with name member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher 08:53:05,944 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-people-oper_testAddReplicasForAllShards: state snapshot applied in 23.47 ms 08:53:05,944 AM [cluster-test-shard-dispatcher-30] [INFO] ShardSnapshotCohort - member-2-shard-people-oper_testAddReplicasForAllShards: Done applying snapshot 08:53:05,945 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#-1109188195] 08:53:05,945 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher#-1670530426] 08:53:05,945 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#580552144] 08:53:05,946 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-pets-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher#580552144] with name member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher 08:53:05,946 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-pets-config_testAddReplicasForAllShards: state snapshot applied in 30.95 ms 08:53:05,946 AM [cluster-test-shard-dispatcher-32] [INFO] ShardSnapshotCohort - member-2-shard-pets-config_testAddReplicasForAllShards: Done applying snapshot 08:53:05,946 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-pets-config_testAddReplicasForAllShards: publishChanges: rootPath: / 08:53:05,946 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 08:53:05,948 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher#-1109188195] with name member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher 08:53:05,948 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-config_testAddReplicasForAllShards: publishChanges: rootPath: / 08:53:05,949 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-config_testAddReplicasForAllShards: state snapshot applied in 29.85 ms 08:53:05,949 AM [cluster-test-shard-dispatcher-27] [INFO] ShardSnapshotCohort - member-2-shard-people-config_testAddReplicasForAllShards: Done applying snapshot 08:53:05,953 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#-359100900] 08:53:05,954 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:53:05,955 AM [cluster-test-shard-dispatcher-14] [INFO] AbstractLeader - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-pets-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:53:05,955 AM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-people-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-people-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:53:05,955 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-pets-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-pets-config_testAddReplicasForAllShards-DTCL-publisher: 9.176 ms 08:53:05,956 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-config_testAddReplicasForAllShards-DTCL-publisher: 7.106 ms 08:53:05,956 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-people-oper_testAddReplicasForAllShards-DTCL-publisher: 9.370 ms 08:53:05,956 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#2015402629] with name member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher 08:53:05,956 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher#2015402629] 08:53:05,956 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddReplicasForAllShards: state snapshot applied in 38.17 ms 08:53:05,956 AM [cluster-test-shard-dispatcher-28] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddReplicasForAllShards: Done applying snapshot 08:53:05,956 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddReplicasForAllShards: publishChanges: rootPath: / 08:53:05,956 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddReplicasForAllShards-DTCL-publisher: 73.63 μs 08:53:05,959 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddReplicasForAllShards: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher#-359100900] with name member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher 08:53:05,959 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddReplicasForAllShards: publishChanges: rootPath: / 08:53:05,959 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddReplicasForAllShards: state snapshot applied in 44.44 ms 08:53:05,959 AM [cluster-test-shard-dispatcher-29] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddReplicasForAllShards: Done applying snapshot 08:53:05,959 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddReplicasForAllShards: Elapsed time for generation of change events for member-2-shard-cars-config_testAddReplicasForAllShards-DTCL-publisher: 84.56 μs 08:53:05,973 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:53:05,977 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-pets-config_testAddReplicasForAllShards} for shard pets from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-pets-config_testAddReplicasForAllShards#1867142625 08:53:05,977 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard pets 08:53:05,977 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-pets-config_testAddReplicasForAllShards 08:53:05,978 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets] 08:53:05,978 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-oper_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-people-oper_testAddReplicasForAllShards#19658949 08:53:05,978 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard people 08:53:05,979 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-oper_testAddReplicasForAllShards 08:53:05,979 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [people] 08:53:05,981 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddReplicasForAllShards (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:53:05,987 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:06,21 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:06,21 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:06,22 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-pets-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-pets-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:06,24 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 08:53:06,24 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:06,26 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-people-config_testAddReplicasForAllShards} for shard people from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-people-config_testAddReplicasForAllShards#820784242 08:53:06,27 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard people 08:53:06,27 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3ab105f5 08:53:06,27 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddReplicasForAllShards/member-1-shard-cars-oper_testAddReplicasForAllShards#-1641144965 08:53:06,27 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 08:53:06,27 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-config_testAddReplicasForAllShards 08:53:06,27 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=pets}, type Config 08:53:06,27 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, people] 08:53:06,27 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 2 08:53:06,27 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:53:06,28 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 3 08:53:06,28 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddReplicasForAllShards 08:53:06,28 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars, people] 08:53:06,28 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 08:53:06,28 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-people-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:06,54 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddReplicasForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddReplicasForAllShards, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddReplicasForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:06,54 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 08:53:06,54 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:06,54 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 08:53:06,54 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:06,76 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:53:06,76 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7cca8cb1 08:53:06,77 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6a150177 08:53:06,77 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 4 08:53:06,77 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:53:06,77 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 5 08:53:06,77 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:53:06,77 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testAddReplicasForAllShards status sync done true 08:53:06,77 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@555287e9 08:53:06,77 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddReplicasForAllShards status sync done true 08:53:06,77 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:53:06,77 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddReplicasForAllShards} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddReplicasForAllShards/member-1-shard-cars-config_testAddReplicasForAllShards#-908843055 08:53:06,77 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 08:53:06,78 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddReplicasForAllShards 08:53:06,78 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [pets, cars, people] 08:53:06,79 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: false, hasLeader : true 08:53:06,79 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:06,85 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e3dfbb7 08:53:06,85 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:53:06,85 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddReplicasForAllShards is ready 08:53:06,85 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 6, actual 6 08:53:06,85 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testAddReplicasForAllShards status sync done true 08:53:06,85 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testAddReplicasForAllShards status sync done true 08:53:06,86 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddReplicasForAllShards status sync done true 08:53:06,98 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1465586145] 08:53:06,98 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:06,100 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#848128064] 08:53:06,101 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-cars-config_testAddReplicasForAllShards#848128064] 08:53:06,102 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1465586145] 08:53:06,102 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:53:06,102 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#1643547117] 08:53:06,102 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-people-config_testAddReplicasForAllShards#1643547117] 08:53:06,103 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards#1465586145] 08:53:06,103 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 08:53:06,103 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#1562675100] 08:53:06,103 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddReplicasForAllShards/member-2-shard-pets-config_testAddReplicasForAllShards#1562675100] 08:53:06,103 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1011249537] 08:53:06,103 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:53:06,103 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-1515580869] 08:53:06,103 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-cars-oper_testAddReplicasForAllShards#-1515580869] 08:53:06,104 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards#-1011249537] 08:53:06,104 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 08:53:06,104 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#809028967] 08:53:06,104 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddReplicasForAllShards/member-2-shard-people-oper_testAddReplicasForAllShards#809028967] 08:53:06,105 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 08:53:06,105 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testAddReplicasForAllShards 08:53:06,105 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddReplicasForAllShards 08:53:06,106 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testAddReplicasForAllShards 08:53:06,106 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 08:53:06,107 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testAddReplicasForAllShards: Stopping Shard 08:53:06,107 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-pets-config_testAddReplicasForAllShards: Stopping Shard 08:53:06,107 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testAddReplicasForAllShards: journal closed 08:53:06,107 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddReplicasForAllShards: journal closed 08:53:06,108 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-config_testAddReplicasForAllShards: Stopping Shard 08:53:06,108 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-people-config_testAddReplicasForAllShards: journal closed 08:53:06,108 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:53:06,109 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:53:06,109 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 08:53:06,109 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddReplicasForAllShards 08:53:06,109 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testAddReplicasForAllShards 08:53:06,110 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-no-leader-oper_testAddReplicasForAllShards 08:53:06,110 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 3 shards 08:53:06,111 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testAddReplicasForAllShards: Stopping Shard 08:53:06,111 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-no-leader-oper_testAddReplicasForAllShards: Stopping Shard 08:53:06,111 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testAddReplicasForAllShards: Stopping Shard 08:53:06,111 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-no-leader-oper_testAddReplicasForAllShards: journal closed 08:53:06,111 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddReplicasForAllShards: journal closed 08:53:06,111 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testAddReplicasForAllShards: journal closed 08:53:06,112 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:53:06,112 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:53:06,115 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:06,115 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:06,141 AM [cluster-test-pekko.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:06,147 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:53:06,174 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddReplicasForAllShards, generation=0} 08:53:06,175 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testAddReplicasForAllShards 08:53:06,176 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddReplicasForAllShards 08:53:06,176 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testAddReplicasForAllShards 08:53:06,176 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 08:53:06,180 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4ec58893 08:53:06,180 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-pets-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4ec58893 08:53:06,180 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 08:53:06,181 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@542f6416 08:53:06,181 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@bf3d708 08:53:06,181 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@bf3d708 08:53:06,181 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@47c51958 08:53:06,181 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 08:53:06,181 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2829e5e1 08:53:06,181 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@42e66bf5 08:53:06,181 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddReplicasForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@42e66bf5 08:53:06,181 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: pauseLeader successfully completed - doing transfer 08:53:07,137 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [7310673253511464277] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:53:10,621 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:10,679 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:11,259 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:11,672 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 08:53:11,674 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:11,674 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:11,705 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:11,901 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:12,219 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:12,538 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:12,718 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:12,860 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:13,177 AM [cluster-test-pekko.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:13,497 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:13,738 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:13,817 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:14,458 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:14,672 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:14,756 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:15,87 AM [cluster-test-shard-dispatcher-14] [WARN] Leader - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 08:53:15,88 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:53:15,88 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:53:15,88 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 08:53:15,88 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 08:53:15,89 AM [cluster-test-shard-dispatcher-29] [WARN] Leader - member-1-shard-cars-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 08:53:15,89 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:53:15,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: leader transfer aborted 08:53:15,90 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddReplicasForAllShards: Failed to transfer leadership in 8.911 s 08:53:15,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: In unpauseLeader 08:53:15,90 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:53:15,90 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 08:53:15,90 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddReplicasForAllShards from Leader to IsolatedLeader 08:53:15,91 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddReplicasForAllShards: Stopping Shard 08:53:15,91 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddReplicasForAllShards: journal closed 08:53:15,95 AM [cluster-test-shard-dispatcher-29] [WARN] Leader - member-1-shard-people-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 08:53:15,95 AM [cluster-test-shard-dispatcher-11] [WARN] Leader - member-1-shard-people-oper_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 08:53:15,96 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-people-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:53:15,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: leader transfer aborted 08:53:15,96 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:53:15,96 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testAddReplicasForAllShards: Failed to transfer leadership in 8.915 s 08:53:15,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-config_testAddReplicasForAllShards: In unpauseLeader 08:53:15,96 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:53:15,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:53:15,96 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 08:53:15,96 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testAddReplicasForAllShards from Leader to IsolatedLeader 08:53:15,96 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 08:53:15,96 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-people-config_testAddReplicasForAllShards: Stopping Shard 08:53:15,96 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testAddReplicasForAllShards from Leader to IsolatedLeader 08:53:15,96 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-1-shard-people-config_testAddReplicasForAllShards: journal closed 08:53:15,98 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:15,416 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: leader transfer aborted 08:53:15,416 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testAddReplicasForAllShards: Failed to transfer leadership in 9.237 s 08:53:15,416 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: In unpauseLeader 08:53:15,417 AM [cluster-test-shard-dispatcher-11] [WARN] Leader - member-1-shard-pets-config_testAddReplicasForAllShards (Leader): At least 1 followers need to be active, Switching member-1-shard-pets-config_testAddReplicasForAllShards from Leader to IsolatedLeader 08:53:15,417 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-pets-config_testAddReplicasForAllShards (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:53:15,417 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testAddReplicasForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:53:15,418 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testAddReplicasForAllShards , received role change from Leader to IsolatedLeader 08:53:15,418 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testAddReplicasForAllShards from Leader to IsolatedLeader 08:53:15,418 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-pets-config_testAddReplicasForAllShards: Stopping Shard 08:53:15,419 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testAddReplicasForAllShards: journal closed 08:53:15,420 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:53:15,420 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:53:15,421 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddReplicasForAllShards, generation=0} 08:53:15,422 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddReplicasForAllShards 08:53:15,422 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testAddReplicasForAllShards 08:53:15,422 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:53:15,423 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testAddReplicasForAllShards: Stopping Shard 08:53:15,423 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddReplicasForAllShards: journal closed 08:53:15,423 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testAddReplicasForAllShards: Stopping Shard 08:53:15,424 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testAddReplicasForAllShards: journal closed 08:53:15,424 AM [cluster-test-pekko.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:53:15,425 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:53:15,428 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:15,428 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:15,431 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:53:15.435] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:15,477 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:53:15,503 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-3734118420326244440] 08:53:15,505 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:53:15,508 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:53:15,508 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:53:15,512 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:15,512 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:15,534 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:53:15,534 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:53:15,535 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:53:15,535 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:15,540 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:15,545 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 08:53:15,550 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1590256594] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,554 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:53:15,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:53:15,562 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:53:15,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStates 08:53:15,563 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStates 08:53:15,563 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:15,563 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:53:15,563 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:53:15,599 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:53:15,601 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@5f17220d} 08:53:15,601 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 08:53:15,604 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:15,605 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:15,605 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStates 08:53:15,606 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:53:15,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:53:15,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates on shard member-1-shard-cars-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:15,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStates 08:53:15,607 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:53:15,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:53:15,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates on shard member-1-shard-people-config_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1141442859] 08:53:15,608 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 08:53:15,608 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,608 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:15,609 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#87333264 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStates 08:53:15,609 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:53:15,609 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates-notifier#87333264] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,611 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 08:53:15,612 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 08:53:15,612 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 08:53:15,612 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#-1505195012 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStates 08:53:15,614 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1141442859] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,614 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates-notifier#-1505195012] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,615 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:53:15,618 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 08:53:15,621 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#752192632] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,625 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:53:15,625 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:53:15,627 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:53:15,629 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:53:15,629 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:15,630 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:53:15,630 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStates 08:53:15,631 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:15,631 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:15,631 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:53:15,633 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:53:15,634 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 08:53:15,635 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 08:53:15,635 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 08:53:15,635 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 08:53:15,636 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@6f6fb7e9} 08:53:15,639 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:15,639 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:15,639 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:15,639 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:15,639 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-1-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#1221921219] 08:53:15,639 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStates 08:53:15,640 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 08:53:15,640 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#1221921219] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,640 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 08:53:15,640 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 08:53:15,640 AM [cluster-test-shard-dispatcher-17] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 08:53:15,641 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#-179460845 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:15,641 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates-notifier#-179460845] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,642 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:53:15,644 AM [cluster-test-shard-dispatcher-15] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStates: Snapshot completed in 9.043 ms, resetting timer for the next recovery snapshot 08:53:15,645 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 08:53:15,646 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 08:53:15,646 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1964267897] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,647 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#1952172427 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStates 08:53:15,647 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:53:15,647 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates-notifier#1952172427] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,648 AM [cluster-test-shard-dispatcher-17] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStates: Snapshot completed in 7.722 ms, resetting timer for the next recovery snapshot 08:53:15,648 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 08:53:15,649 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:53:15,649 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:53:15,649 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-1-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1964267897] 08:53:15,655 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:53:15,656 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:15,656 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:15,656 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 08:53:15,656 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:53:15,656 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStates is initialized 08:53:15,656 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 08:53:15,656 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from null to Follower 08:53:15,657 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:53:15,660 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:53:15,661 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:53:15,661 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStates is initialized 08:53:15,661 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:15,664 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:15,665 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 08:53:15,665 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 08:53:15,665 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from null to Follower 08:53:15,665 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 08:53:15,665 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 08:53:15,665 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 08:53:15,666 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 08:53:15,666 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 08:53:15,666 AM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 08:53:15,668 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStates: Snapshot completed in 3.053 ms, resetting timer for the next recovery snapshot 08:53:15,669 AM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStates: Snapshot completed in 3.253 ms, resetting timer for the next recovery snapshot 08:53:15,674 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:15,674 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:15,674 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:53:15,674 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStates is initialized 08:53:15,675 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 08:53:15,675 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 08:53:15,675 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from null to Follower 08:53:15,675 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:15,675 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:15,675 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:53:15,675 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStates is initialized 08:53:15,675 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 08:53:15,675 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 08:53:15,675 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from null to Follower 08:53:15,684 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-4114085753308370643] 08:53:15,685 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 08:53:15,687 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:15,687 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:53:15,694 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:15,695 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:15,709 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:15,713 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:15,717 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 08:53:15,729 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#579243638] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,733 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:53:15,733 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:53:15,755 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:53:15,755 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStates 08:53:15,756 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStates 08:53:15,758 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:53:15,761 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@6e7112c0} 08:53:15,761 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 08:53:15,762 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:15,762 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStates 08:53:15,762 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:15,764 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:15,768 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 08:53:15,769 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStates 08:53:15,770 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-27428110] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,770 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1922627754] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,770 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 08:53:15,771 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:53:15,771 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#1316403556 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStates 08:53:15,771 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates-notifier#1316403556] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,772 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:53:15,772 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:53:15,772 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 08:53:15,772 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1731617910] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,774 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 08:53:15,774 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:53:15,774 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#2142937761 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStates 08:53:15,774 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates-notifier#2142937761] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,775 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:53:15,776 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 08:53:15,777 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 08:53:15,777 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 08:53:15,777 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 08:53:15,778 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:53:15,779 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStates: Snapshot completed in 2.591 ms, resetting timer for the next recovery snapshot 08:53:15,783 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 08:53:15,783 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 08:53:15,783 AM [cluster-test-shard-dispatcher-27] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 08:53:15,787 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:53:15,787 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStates is initialized 08:53:15,787 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:53:15,788 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStates 08:53:15,788 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStates 08:53:15,793 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:15,793 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:15,793 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:53:15,793 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 08:53:15,793 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 08:53:15,794 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStates from null to Follower 08:53:15,795 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 08:53:15,795 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@50024bc6} 08:53:15,796 AM [cluster-test-shard-dispatcher-27] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStates: Snapshot completed in 13.06 ms, resetting timer for the next recovery snapshot 08:53:15,797 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:15,797 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStates 08:53:15,799 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 08:53:15,799 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1215126815] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,800 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#1291170643 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStates 08:53:15,800 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:53:15,800 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates-notifier#1291170643] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,801 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStates 08:53:15,802 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1685495501] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,804 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 08:53:15,805 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 08:53:15,805 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#-5049810 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStates 08:53:15,806 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:53:15,806 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStates is initialized 08:53:15,806 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:15,806 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:15,806 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 08:53:15,806 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 08:53:15,807 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:53:15,807 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates-notifier#-5049810] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,807 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from null to Follower 08:53:15,808 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:53:15,809 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 08:53:15,811 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:53:15,818 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 08:53:15,818 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 08:53:15,818 AM [cluster-test-shard-dispatcher-27] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 08:53:15,821 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 08:53:15,821 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 08:53:15,821 AM [cluster-test-shard-dispatcher-30] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 08:53:15,822 AM [cluster-test-shard-dispatcher-27] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStates: Snapshot completed in 4.509 ms, resetting timer for the next recovery snapshot 08:53:15,828 AM [cluster-test-shard-dispatcher-30] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStates: Snapshot completed in 6.663 ms, resetting timer for the next recovery snapshot 08:53:15,828 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:15,828 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:15,828 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:53:15,828 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStates is initialized 08:53:15,830 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:53:15,832 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 08:53:15,832 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 08:53:15,832 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from null to Follower 08:53:15,833 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:15,833 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:15,834 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:53:15,834 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStates is initialized 08:53:15,834 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 08:53:15,834 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 08:53:15,834 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:53:15,837 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStates from null to Follower 08:53:15,838 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:15,838 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:15,838 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:53:15,838 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:15,838 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:53:15,838 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:53:15,838 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:53:15,838 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:53:15,838 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:53:15,838 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:53:15,838 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:53:15,838 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:53:15,838 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:53:15,838 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:15,839 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:53:15,853 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-2128653241233985843] 08:53:15,853 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 08:53:15,859 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:15,859 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:53:15,861 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:15,862 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:15,878 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:15,881 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:15,885 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStates 08:53:15,893 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-2005274068] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,894 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:53:15,894 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:53:15,913 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:53:15,914 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStates 08:53:15,914 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStates 08:53:15,915 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:53:15,916 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:53:15,917 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@3e410c33} 08:53:15,917 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} started 08:53:15,917 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:15,917 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStates 08:53:15,917 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:15,927 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:15,936 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStates 08:53:15,937 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:53:15,940 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStates 08:53:15,941 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2001896033] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,941 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Shard created, persistent : true 08:53:15,944 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#1817161897] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,944 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates-notifier#1817161897 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStates 08:53:15,944 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:53:15,946 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStates: journal open: applyTo=0 08:53:15,949 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-637875673] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,949 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Shard created, persistent : true 08:53:15,950 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#973458348 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStates 08:53:15,950 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:53:15,950 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates-notifier#973458348] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,951 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStates: journal open: applyTo=0 08:53:15,954 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#834736662] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:15,956 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:53:15,957 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:53:15,958 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:53:15,958 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:15,959 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:53:15,959 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:53:15,959 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:53:15,959 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:53:15,959 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 08:53:15,959 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 08:53:15,959 AM [cluster-test-shard-dispatcher-26] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 08:53:15,962 AM [cluster-test-shard-dispatcher-26] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStates: Snapshot completed in 2.727 ms, resetting timer for the next recovery snapshot 08:53:15,964 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:53:15,964 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:53:15,968 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:15,969 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStates 08:53:15,970 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:15,970 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:15,970 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:53:15,998 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 08:53:15,999 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 08:53:15,999 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStates: Taking snapshot on entry with index -1 08:53:16,2 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,2 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStates: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:53:16,2 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:53:16,2 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStates is initialized 08:53:16,2 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,2 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from null to Follower 08:53:16,2 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:53:16,3 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 08:53:16,3 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@3c3f30d1} 08:53:16,3 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from null to Follower 08:53:16,4 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} started 08:53:16,4 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:53:16,4 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:16,4 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:16,5 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStates: Snapshot completed in 6.608 ms, resetting timer for the next recovery snapshot 08:53:16,6 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:16,6 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:16,6 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates on shard member-3-shard-cars-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1181168834] 08:53:16,6 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStates 08:53:16,8 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Shard created, persistent : true 08:53:16,9 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:53:16,9 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:53:16,9 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStates on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates on shard member-3-shard-people-oper_testFlipMemberVotingStates whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#728478374] 08:53:16,9 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:53:16,9 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1181168834] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:16,10 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Shard created, persistent : true 08:53:16,10 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#-551117857 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:16,10 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#728478374] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:16,10 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#1496508347 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStates 08:53:16,10 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStates: Starting recovery with journal batch size 1 08:53:16,11 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates-notifier#-551117857] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:16,11 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates-notifier#1496508347] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:16,12 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStates: journal open: applyTo=0 08:53:16,12 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStates: journal open: applyTo=0 08:53:16,15 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:53:16,15 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,15 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,15 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from null to Follower 08:53:16,15 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:53:16,15 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:53:16,15 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStates is initialized 08:53:16,16 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates 08:53:16,15 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStates: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:53:16,16 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from null to Follower 08:53:16,20 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 08:53:16,20 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 08:53:16,20 AM [cluster-test-shard-dispatcher-26] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 08:53:16,21 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStates: Local TermInfo store seeded with TermInfo{term=1} 08:53:16,21 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStates: taking snapshot to clear Pekko persistence to 2 08:53:16,21 AM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStates: Taking snapshot on entry with index -1 08:53:16,23 AM [cluster-test-shard-dispatcher-26] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStates: Snapshot completed in 2.992 ms, resetting timer for the next recovery snapshot 08:53:16,29 AM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStates: Snapshot completed in 8.358 ms, resetting timer for the next recovery snapshot 08:53:16,30 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:53:16,30 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStates is initialized 08:53:16,30 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,30 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,31 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:16,31 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from null to Follower 08:53:16,31 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 08:53:16,31 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from null to Follower 08:53:16,34 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:53:16,34 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStates is initialized 08:53:16,34 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,34 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,34 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from null to Follower 08:53:16,35 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:53:16,35 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates 08:53:16,35 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from null to Follower 08:53:16,523 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:53:16,523 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:53:16,524 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:16,524 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:16,524 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:16,524 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:53:16,524 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-18] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:16,525 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:53:16,525 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:53:16,526 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:16,526 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:53:16,526 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates 08:53:16,526 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:53:16,526 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:53:16,543 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:53:16,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:53:16,544 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:16,544 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:53:16,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:53:16,544 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:53:16,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:53:16,544 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:53:16,544 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:53:16,724 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 08:53:16,724 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:53:16,724 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,724 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,724 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:16,724 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 08:53:16,737 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:16,742 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStates with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:53:16,742 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates 08:53:16,743 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStates: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStates set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates 08:53:16,746 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 08:53:16,746 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:53:16,746 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,746 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,746 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:16,746 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 08:53:16,748 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:53:16,755 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 2 08:53:16,755 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:53:16,755 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,755 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,756 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:16,756 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 08:53:16,757 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:53:16,758 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 2 08:53:16,759 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:53:16,759 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,759 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,759 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:16,761 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 08:53:16,762 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:53:16,771 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:53:16,771 AM [cluster-test-shard-dispatcher-20] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:53:16,771 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:53:16,773 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:53:16,773 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:16,773 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:16,773 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@70741b6b 08:53:16,773 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 08:53:16,773 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to PreLeader 08:53:16,775 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:53:16,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:16,775 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:16,775 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 08:53:16,775 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4851e94a 08:53:16,776 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to PreLeader 08:53:16,775 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:53:16,777 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:16,777 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:16,779 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 08:53:16,780 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 08:53:16,780 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to PreLeader 08:53:16,780 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@87a1a1f 08:53:16,780 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to PreLeader 08:53:16,781 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:53:16,781 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:53:16,781 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:16,781 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:16,781 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to PreLeader 08:53:16,782 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 08:53:16,782 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7492a18e 08:53:16,782 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to PreLeader 08:53:16,783 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done false 08:53:16,784 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7108f2a5 08:53:16,784 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4a8c22fb 08:53:16,784 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:53:16,784 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done false 08:53:16,784 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStates status sync done true 08:53:16,785 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 08:53:16,785 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@9ee0958 08:53:16,786 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 08:53:16,786 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 08:53:16,786 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 08:53:16,786 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6cfc0665 08:53:16,787 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2a4fb124 08:53:16,787 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStates status sync done true 08:53:16,787 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStates (Follower): Term 2 in "AppendEntries{term=2, leaderId=member-1-shard-people-config_testFlipMemberVotingStates, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 1 - updating term 08:53:16,790 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done false 08:53:16,790 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStates status sync done true 08:53:16,790 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 08:53:16,790 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 08:53:16,790 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2036fecf 08:53:16,790 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:53:16,790 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@78303 08:53:16,790 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:53:16,792 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 08:53:16,795 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 08:53:16,795 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@8c0f824 08:53:16,795 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:53:16,795 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 08:53:16,795 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 08:53:16,800 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:53:16,800 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:16,800 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 08:53:16,800 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from PreLeader to Leader 08:53:16,801 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:53:16,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:16,801 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 08:53:16,801 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from PreLeader to Leader 08:53:16,802 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:53:16,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:16,802 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from PreLeader to Leader 08:53:16,803 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from PreLeader to Leader 08:53:16,803 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:53:16,808 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:53:16,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:16,808 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from PreLeader to Leader 08:53:16,808 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from PreLeader to Leader 08:53:16,808 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:53:16,808 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 08:53:16,808 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:53:16,808 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 08:53:16,808 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:53:16,808 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} is now ready 08:53:16,808 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:53:16,808 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} is now ready 08:53:16,809 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1590256594] 08:53:16,809 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:16,809 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:16,809 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:16,812 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 08:53:16,812 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 08:53:16,812 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 08:53:16,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:53:16,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:53:16,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:53:16,812 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:53:16,817 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:53:16,818 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:53:16,819 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:16,819 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:53:16,819 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=true, member-1-shard-people-oper_testFlipMemberVotingStates=false, member-2-shard-people-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:53:16,819 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=false, member-1-shard-cars-oper_testFlipMemberVotingStates=false, member-3-shard-cars-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:16,821 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=true, member-2-shard-cars-config_testFlipMemberVotingStates=false, member-1-shard-cars-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:53:16,821 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=false, member-2-shard-people-config_testFlipMemberVotingStates=false, member-3-shard-people-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:53:16,843 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:53:16,843 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:53:16,844 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 08:53:16,845 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:53:16,846 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:53:16,846 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3cb3758c 08:53:16,846 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6b03f3a9 08:53:16,846 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6b03f3a9 08:53:16,846 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:16,846 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3c293eb9 08:53:16,846 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@ce0274e 08:53:16,846 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3c293eb9 08:53:16,846 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:16,847 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:53:16,847 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:53:16,847 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7ae1b89b 08:53:16,847 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1a2aed36 08:53:16,847 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7ae1b89b 08:53:16,847 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1a2aed36 08:53:16,847 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:16,847 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:16,848 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@469a11dc 08:53:16,848 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1590256594] 08:53:16,853 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@11abc2fe 08:53:16,853 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@69a6faf1 08:53:16,856 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2edd5c0a 08:53:16,856 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:53:16,856 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@300105fa 08:53:16,857 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:53:16,861 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@69559727 08:53:16,861 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1ae71211 08:53:16,861 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6135112 08:53:16,861 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3c77a8bb 08:53:16,863 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:53:16,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:53:16,868 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3c08d3a4 08:53:16,869 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:16,869 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:16,869 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:16,871 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1590256594] 08:53:16,871 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:53:16,871 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1141442859] 08:53:16,871 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1141442859] 08:53:16,871 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#752192632] 08:53:16,871 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:53:16,872 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#1221921219] 08:53:16,872 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#1221921219] 08:53:16,872 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#752192632] 08:53:16,872 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:53:16,872 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1964267897] 08:53:16,872 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1964267897] 08:53:16,873 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#579243638] 08:53:16,874 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:16,874 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1922627754] 08:53:16,874 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1922627754] 08:53:16,874 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:53:16,875 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#579243638] 08:53:16,875 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:53:16,875 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1731617910] 08:53:16,876 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1731617910] 08:53:16,876 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-27428110] 08:53:16,876 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:53:16,876 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1215126815] 08:53:16,876 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1215126815] 08:53:16,876 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-27428110] 08:53:16,876 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 08:53:16,876 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1685495501] 08:53:16,876 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1685495501] 08:53:16,877 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-2005274068] 08:53:16,877 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:16,877 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2001896033] 08:53:16,877 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2001896033] 08:53:16,882 AM [cluster-test-shard-dispatcher-27] [INFO] Candidate - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 08:53:16,882 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 08:53:16,882 AM [cluster-test-shard-dispatcher-29] [INFO] Candidate - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 08:53:16,882 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,882 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,882 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 08:53:16,882 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,882 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,882 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:16,882 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:16,883 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1f3a32a6 08:53:16,883 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 08:53:16,883 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2193ccd1 08:53:16,883 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 08:53:16,883 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:16,883 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:16,883 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:16,884 AM [cluster-test-shard-dispatcher-26] [INFO] Candidate - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 3 08:53:16,884 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 08:53:16,884 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 08:53:16,884 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,884 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,884 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:16,885 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:16,885 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:16,885 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:16,885 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:16,885 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:16,885 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@701dc1ef 08:53:16,885 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2f542693 08:53:16,885 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-2005274068] 08:53:16,885 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-3-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 3 08:53:16,885 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 08:53:16,888 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 3 08:53:16,885 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 08:53:16,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,889 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e32cc6f 08:53:16,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,889 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 08:53:16,889 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6acbf8af 08:53:16,889 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:53:16,889 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 08:53:16,889 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:16,889 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-637875673] 08:53:16,889 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@52dd1ed9 08:53:16,889 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 08:53:16,889 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:16,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:16,889 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:16,889 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 08:53:16,889 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@792d85a8 08:53:16,890 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 08:53:16,890 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:53:16,885 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:16,890 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 08:53:16,890 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-637875673] 08:53:16,890 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 08:53:16,890 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:53:16,891 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#834736662] 08:53:16,893 AM [cluster-test-shard-dispatcher-20] [INFO] AbstractLeader - member-1-shard-people-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 08:53:16,893 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 08:53:16,894 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 08:53:16,894 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 08:53:16,894 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:53:16,894 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStates (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 2 - updating term 08:53:16,894 AM [cluster-test-shard-dispatcher-31] [INFO] AbstractLeader - member-1-shard-cars-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 08:53:16,895 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1181168834] 08:53:16,895 AM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 08:53:16,895 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1181168834] 08:53:16,895 AM [cluster-test-shard-dispatcher-18] [INFO] AbstractLeader - member-1-shard-people-config_testFlipMemberVotingStates (Leader): Term 3 in "AppendEntries{term=3, leaderId=member-3-shard-people-config_testFlipMemberVotingStates, prevLogIndex=1, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 2 - switching to Follower 08:53:16,896 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#834736662] 08:53:16,896 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:53:16,897 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#728478374] 08:53:16,897 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#728478374] 08:53:16,897 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1590256594] 08:53:16,897 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:16,897 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:16,898 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:16,941 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 08:53:16,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,943 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 08:53:16,943 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,943 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,943 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7255ea61 08:53:16,944 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 08:53:16,944 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 08:53:16,944 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 08:53:16,944 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 08:53:16,944 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2952bcb 08:53:16,944 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,944 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,944 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2fcf675e 08:53:16,945 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 08:53:16,945 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 08:53:16,945 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 08:53:16,945 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 3 08:53:16,945 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 08:53:16,945 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:16,945 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:16,945 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7618532b 08:53:16,945 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@8088f27 08:53:16,945 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 08:53:16,945 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Leader to Follower 08:53:16,945 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@74ce3ed6 08:53:16,945 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 08:53:16,945 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:53:16,948 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@10be7a8b 08:53:16,948 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 08:53:16,948 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5310e693 08:53:16,948 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:53:16,948 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 08:53:16,954 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 08:53:16,955 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done true 08:53:16,955 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done true 08:53:16,955 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 08:53:16,995 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:17,46 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:17,97 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:17,148 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:17,198 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:17,198 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader changed to member-3-shard-people-oper_testFlipMemberVotingStates 08:53:17,198 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-oper_testFlipMemberVotingStates in 352.3 ms 08:53:17,199 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@689307eb 08:53:17,199 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done false 08:53:17,205 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-people-oper_testFlipMemberVotingStates status sync done true 08:53:17,206 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:17,206 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-oper_testFlipMemberVotingStates in 359.3 ms 08:53:17,206 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 08:53:17,206 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2f5ef303 08:53:17,207 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:53:17,208 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader changed to member-3-shard-cars-config_testFlipMemberVotingStates 08:53:17,208 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader changed to member-3-shard-people-config_testFlipMemberVotingStates 08:53:17,208 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-people-config_testFlipMemberVotingStates in 360.8 ms 08:53:17,208 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-3-shard-cars-config_testFlipMemberVotingStates in 361.6 ms 08:53:17,208 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done false 08:53:17,208 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 08:53:17,208 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@37185489 08:53:17,208 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@39dfac1a 08:53:17,208 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:53:17,210 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 08:53:17,212 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 08:53:17,212 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testFlipMemberVotingStates status sync done true 08:53:17,249 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:17,250 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#752192632] 08:53:17,250 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:53:17,250 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#1221921219] 08:53:17,251 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#1221921219] 08:53:17,251 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 08:53:17,251 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStates 08:53:17,251 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStates 08:53:17,251 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:53:17,251 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:53:17,252 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:53:17,253 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:53:17,253 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:17,253 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:53:17,253 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:53:17,253 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:53:17,255 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-people-config_testFlipMemberVotingStates=true, member-2-shard-people-config_testFlipMemberVotingStates=true, member-3-shard-people-config_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:53:17,255 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-people-oper_testFlipMemberVotingStates=false, member-1-shard-people-oper_testFlipMemberVotingStates=true, member-2-shard-people-oper_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:53:17,255 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testFlipMemberVotingStates=false, member-2-shard-cars-config_testFlipMemberVotingStates=true, member-1-shard-cars-config_testFlipMemberVotingStates=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:53:17,256 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStates=true, member-1-shard-cars-oper_testFlipMemberVotingStates=true, member-3-shard-cars-oper_testFlipMemberVotingStates=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-people-config_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-3-shard-cars-oper_testFlipMemberVotingStates} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:53:17,258 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 08:53:17,259 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1590256594] 08:53:17,259 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:17,259 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:17,260 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:17,262 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorVotingConfigSupport - member-3-shard-people-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:53:17,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5cb414b1 08:53:17,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-3-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5cb414b1 08:53:17,263 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:17,264 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@9000b27 08:53:17,267 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1750ea3d 08:53:17,268 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:53:17,268 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorVotingConfigSupport - member-3-shard-people-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:53:17,268 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e692ca8 08:53:17,269 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e692ca8 08:53:17,269 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:17,268 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorVotingConfigSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:53:17,269 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4b239a90 08:53:17,269 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4b239a90 08:53:17,269 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:17,269 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorVotingConfigSupport - member-3-shard-cars-config_testFlipMemberVotingStates: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStates, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStates, isVoting=false]]} has been successfully replicated to a majority of followers 08:53:17,269 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@51a390f0 08:53:17,269 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@51a390f0 08:53:17,269 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:17,270 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5f4d44e9 08:53:17,270 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4c51afd4 08:53:17,270 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@296da60e 08:53:17,271 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@18cd596e 08:53:17,273 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5b068b74 08:53:17,274 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@296329e7 08:53:17,276 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@39cca715 08:53:17,276 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@758003d2 08:53:17,277 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@31ced4c1 08:53:17,277 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1e71a00 08:53:17,278 AM [cluster-test-shard-dispatcher-31] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 08:53:17,279 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 08:53:17,279 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:17,279 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:17,279 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:17,279 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4dfe6586 08:53:17,279 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Follower to Candidate 08:53:17,281 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 08:53:17,281 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:53:17,282 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:53:17,282 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:53:17,289 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 08:53:17,289 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:17,289 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:17,289 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 08:53:17,289 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@31bad80 08:53:17,289 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStates from Candidate to Leader 08:53:17,290 AM [cluster-test-shard-dispatcher-31] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 08:53:17,290 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 08:53:17,290 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:17,290 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:17,290 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:17,290 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2bae441 08:53:17,290 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Follower to Candidate 08:53:17,291 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-3-shard-people-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-1-shard-people-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 08:53:17,291 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 08:53:17,291 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done false 08:53:17,292 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@596dcfc0 08:53:17,294 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-people-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 08:53:17,294 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:17,294 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:17,294 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStates , received role change from Leader to Follower 08:53:17,295 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@72dedb41 08:53:17,295 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStates from Leader to Follower 08:53:17,296 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStates status sync done true 08:53:17,297 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-2-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 4 08:53:17,298 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 08:53:17,298 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:17,298 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:17,298 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:17,298 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7f577f13 08:53:17,298 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 08:53:17,298 AM [cluster-test-shard-dispatcher-27] [INFO] Candidate - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 4 08:53:17,298 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-2-shard-cars-config_testFlipMemberVotingStates (Follower): Cannot append entries because sender's term 3 is less than 4 08:53:17,298 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 4 08:53:17,298 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:17,299 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:17,299 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:17,299 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@68426240 08:53:17,299 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 08:53:17,299 AM [cluster-test-shard-dispatcher-18] [INFO] Follower - member-1-shard-people-config_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 08:53:17,300 AM [cluster-test-shard-dispatcher-31] [INFO] AbstractLeader - member-3-shard-cars-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntriesReply{term=4, success=false, followerId=member-2-shard-cars-config_testFlipMemberVotingStates, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}" message is greater than leader's term 3 - switching to Follower 08:53:17,300 AM [cluster-test-shard-dispatcher-17] [INFO] Follower - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than follower's term 3 - updating term 08:53:17,300 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 08:53:17,300 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:17,301 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:17,301 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStates , received role change from Candidate to Leader 08:53:17,301 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6ef7178b 08:53:17,301 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStates from Candidate to Leader 08:53:17,302 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@49207c52 08:53:17,303 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done false 08:53:17,304 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-3-shard-cars-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 08:53:17,304 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:17,304 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:17,305 AM [cluster-test-shard-dispatcher-31] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: leader changed to member-1-shard-cars-config_testFlipMemberVotingStates 08:53:17,305 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStates , received role change from Leader to Follower 08:53:17,305 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-config_testFlipMemberVotingStates in 35.32 ms 08:53:17,305 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@e46d3e7 08:53:17,305 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStates from Leader to Follower 08:53:17,305 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@432ff6cf 08:53:17,305 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done false 08:53:17,306 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStates status sync done true 08:53:17,309 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 08:53:17,309 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:17,309 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:17,310 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 08:53:17,310 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:17,310 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:17,310 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 08:53:17,310 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@62aebdd1 08:53:17,310 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 08:53:17,310 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ac0c5d7 08:53:17,310 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 08:53:17,310 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:53:17,310 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 08:53:17,310 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:53:17,311 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:17,311 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1590256594] 08:53:17,311 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done false 08:53:17,312 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:53:17,312 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done false 08:53:17,312 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1141442859] 08:53:17,312 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@34e4b17b 08:53:17,312 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@554668f5 08:53:17,312 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-people-config_testFlipMemberVotingStates#1141442859] 08:53:17,312 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:53:17,312 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:53:17,312 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#752192632] 08:53:17,312 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:53:17,312 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#1221921219] 08:53:17,313 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-cars-oper_testFlipMemberVotingStates#1221921219] 08:53:17,313 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-3-shard-people-config_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-people-config_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 08:53:17,314 AM [cluster-test-shard-dispatcher-31] [INFO] AbstractLeader - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 4 in "AppendEntries{term=4, leaderId=member-2-shard-cars-oper_testFlipMemberVotingStates, prevLogIndex=2, prevLogTerm=2, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than leader's term 3 - switching to Follower 08:53:17,314 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#752192632] 08:53:17,315 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:53:17,315 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1964267897] 08:53:17,315 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-1-shard-people-oper_testFlipMemberVotingStates#1964267897] 08:53:17,315 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStates status sync done true 08:53:17,315 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#579243638] 08:53:17,315 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:17,315 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1922627754] 08:53:17,315 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-cars-config_testFlipMemberVotingStates#1922627754] 08:53:17,316 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-people-config_testFlipMemberVotingStates status sync done true 08:53:17,316 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1 Received follower initial sync status for member-1-shard-cars-oper_testFlipMemberVotingStates status sync done true 08:53:17,316 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#579243638] 08:53:17,316 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:53:17,316 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1731617910] 08:53:17,316 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-2-shard-people-config_testFlipMemberVotingStates#1731617910] 08:53:17,317 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-27428110] 08:53:17,317 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActorBehavior - member-3-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 08:53:17,317 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-3-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 4 08:53:17,317 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:53:17,317 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:17,317 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:17,317 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:17,317 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:17,317 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 08:53:17,317 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 08:53:17,317 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1215126815] 08:53:17,317 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@76f0884f 08:53:17,317 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@31706bba 08:53:17,317 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-cars-oper_testFlipMemberVotingStates#1215126815] 08:53:17,317 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 08:53:17,317 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStates from Leader to Follower 08:53:17,317 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#-27428110] 08:53:17,317 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 08:53:17,317 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1685495501] 08:53:17,317 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-2-shard-people-oper_testFlipMemberVotingStates#1685495501] 08:53:17,318 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-2005274068] 08:53:17,318 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:17,318 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2001896033] 08:53:17,318 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-cars-config_testFlipMemberVotingStates#2001896033] 08:53:17,318 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-2005274068] 08:53:17,318 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:53:17,319 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-637875673] 08:53:17,319 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-3-shard-people-config_testFlipMemberVotingStates#-637875673] 08:53:17,319 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#834736662] 08:53:17,319 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:53:17,319 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1181168834] 08:53:17,319 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-cars-oper_testFlipMemberVotingStates#1181168834] 08:53:17,319 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates#834736662] 08:53:17,319 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:53:17,319 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#728478374] 08:53:17,319 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStates/member-3-shard-people-oper_testFlipMemberVotingStates#728478374] 08:53:17,320 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates#-1590256594] 08:53:17,320 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:17,320 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:17,320 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStates/member-1-shard-cars-config_testFlipMemberVotingStates#1168035458] 08:53:17,321 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:53:17,321 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStates 08:53:17,322 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStates 08:53:17,322 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 08:53:17,322 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 08:53:17,322 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStates: Stopping Shard 08:53:17,323 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStates: journal closed 08:53:17,323 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStates: journal closed 08:53:17,323 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:53:17,323 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:53:17,323 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:53:17,324 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStates 08:53:17,324 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStates 08:53:17,324 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:53:17,324 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 08:53:17,324 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 08:53:17,324 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStates: journal closed 08:53:17,325 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStates: journal closed 08:53:17,325 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:53:17,325 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:53:17,327 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:17,327 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:17,334 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:17,335 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:17,336 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:17,336 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:17,338 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:53:17,354 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:53:17,355 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStates 08:53:17,356 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStates 08:53:17,356 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:53:17,356 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@28590e10 08:53:17,357 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7ee804ac 08:53:17,357 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7ee804ac 08:53:17,357 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:17,358 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 08:53:17,359 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStates: journal closed 08:53:17,361 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@378aed2d 08:53:17,370 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:53:17,377 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStates (Candidate): Starting new election term 5 08:53:17,378 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 08:53:17,378 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:17,378 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:17,378 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:17,378 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3c4659ce 08:53:17,378 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Follower to Candidate 08:53:17,380 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-2-shard-people-config_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-people-config_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 08:53:17,384 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-people-config_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 08:53:17,384 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:17,384 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:17,385 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStates , received role change from Leader to Follower 08:53:17,385 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ef43687 08:53:17,385 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStates from Leader to Follower 08:53:17,386 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 08:53:17,386 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:17,386 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:17,386 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStates , received role change from Candidate to Leader 08:53:17,386 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@29c74b83 08:53:17,386 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStates from Candidate to Leader 08:53:17,386 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStates is ready 08:53:17,388 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: leader changed to member-1-shard-people-config_testFlipMemberVotingStates 08:53:17,388 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-people-config_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-people-config_testFlipMemberVotingStates in 31.62 ms 08:53:17,388 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStates status sync done false 08:53:17,389 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStates: Stopping Shard 08:53:17,389 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStates: journal closed 08:53:17,390 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:53:17,390 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:53:17,390 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:53:17,391 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStates 08:53:17,391 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStates 08:53:17,391 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:53:17,392 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7950e356 08:53:17,392 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7950e356 08:53:17,392 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:17,392 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@53723939 08:53:17,392 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 08:53:17,392 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStates: journal closed 08:53:17,394 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@626f946b 08:53:17,396 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader transfer complete - waiting for new leader 08:53:17,402 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate): Starting new election term 5 08:53:17,402 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Follower) :- Switching from behavior Follower to Candidate, election term: 5 08:53:17,402 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:17,402 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:17,403 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Follower to Candidate 08:53:17,403 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@73118e34 08:53:17,403 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Follower to Candidate 08:53:17,404 AM [cluster-test-shard-dispatcher-27] [INFO] AbstractLeader - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStates, lastLogIndex=3, lastLogTerm=3}" message is greater than leader's term 4 - switching to Follower 08:53:17,411 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-2-shard-cars-oper_testFlipMemberVotingStates (Leader) :- Switching from behavior Leader to Follower, election term: 5 08:53:17,411 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: false, hasLeader : false 08:53:17,411 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:17,411 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStates , received role change from Leader to Follower 08:53:17,412 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@19fa1de9 08:53:17,412 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStates from Leader to Follower 08:53:17,414 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStates (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 08:53:17,414 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStates: onLeadershipChange, isLeader: true, hasLeader : true 08:53:17,414 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: became leader with frontend state for [] 08:53:17,414 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStates , received role change from Candidate to Leader 08:53:17,414 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c31100c 08:53:17,414 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStates from Candidate to Leader 08:53:17,414 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:53:17,415 AM [cluster-test-shard-dispatcher-27] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: leader changed to member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:17,415 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3dbbb488 08:53:17,415 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-oper_testFlipMemberVotingStates: Successfully transferred leadership to member-1-shard-cars-oper_testFlipMemberVotingStates in 24.01 ms 08:53:17,416 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStates is ready 08:53:17,416 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStates status sync done false 08:53:17,416 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 08:53:17,417 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStates: journal closed 08:53:17,417 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:53:17,417 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:53:17,419 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:17,420 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:17,541 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:53:18,341 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-2128653241233985843] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:53:18,444 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-4114085753308370643] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:53:18,444 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:18,445 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:18,447 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:53:18.452] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:18,460 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStates, generation=0} 08:53:18,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStates 08:53:18,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStates 08:53:18,461 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:53:18,462 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1312f6fe 08:53:18,462 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1312f6fe 08:53:18,462 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6173f87d 08:53:18,462 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:18,462 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6173f87d 08:53:18,462 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:18,463 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1d14557b 08:53:18,463 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@346467e 08:53:19,542 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: leader transfer aborted 08:53:19,542 AM [cluster-test-shard-dispatcher-18] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.080 s 08:53:19,542 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStates: In unpauseLeader 08:53:19,543 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStates: Stopping Shard 08:53:19,544 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStates: journal closed 08:53:19,571 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: leader transfer aborted 08:53:19,571 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStates: Failed to transfer leadership in 1.109 s 08:53:19,571 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: In unpauseLeader 08:53:19,572 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStates: Stopping Shard 08:53:19,572 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStates: journal closed 08:53:19,573 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:53:19,573 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:53:19,574 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStates, generation=0} 08:53:19,574 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStates 08:53:19,575 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStates 08:53:19,575 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:53:19,575 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@42be58e0 08:53:19,575 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c59080c 08:53:19,575 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c59080c 08:53:19,575 AM [cluster-test-shard-dispatcher-18] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:19,576 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7878a47f 08:53:19,576 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@37555df6 08:53:19,576 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStates: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@37555df6 08:53:19,576 AM [cluster-test-shard-dispatcher-20] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: pauseLeader successfully completed - doing transfer 08:53:20,641 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: leader transfer aborted 08:53:20,641 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.066 s 08:53:20,648 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: In unpauseLeader 08:53:20,649 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStates: Stopping Shard 08:53:20,649 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStates: journal closed 08:53:20,650 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: leader transfer aborted 08:53:20,650 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStates: Failed to transfer leadership in 1.075 s 08:53:20,650 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: In unpauseLeader 08:53:20,651 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStates: Stopping Shard 08:53:20,651 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStates: journal closed 08:53:20,652 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:53:20,652 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:53:20,654 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:20,654 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:20,660 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:53:20.663] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:20,689 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:53:20,700 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-4128572270553110816] 08:53:20,701 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:53:20,704 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:53:20,704 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:53:20,705 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:20,705 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:20,723 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:53:20,723 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:53:20,723 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:53:20,723 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:20,725 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:20,728 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 08:53:20,734 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:53:20,734 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:53:20,736 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-940410098] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:20,737 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:53:20,737 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testModuleShardLeaderMovement 08:53:20,738 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testModuleShardLeaderMovement 08:53:20,738 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:20,738 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 08:53:20,738 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 08:53:20,746 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 08:53:20,747 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@287b76fe} 08:53:20,747 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 08:53:20,749 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:20,749 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testModuleShardLeaderMovement 08:53:20,753 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 08:53:20,753 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 08:53:20,753 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement on shard member-1-shard-cars-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223] 08:53:20,753 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testModuleShardLeaderMovement 08:53:20,754 AM [cluster-test-shard-dispatcher-10] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 08:53:20,754 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 08:53:20,754 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement on shard member-1-shard-people-config_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-1217100457] 08:53:20,754 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 08:53:20,755 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:53:20,757 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:20,757 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#20765021 created and ready for shard:member-1-shard-cars-config_testModuleShardLeaderMovement 08:53:20,757 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement-notifier#20765021] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:20,764 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 08:53:20,765 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Shard created, persistent : true 08:53:20,765 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement#-1217100457] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:20,766 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:20,766 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#1719464469 created and ready for shard:member-1-shard-people-config_testModuleShardLeaderMovement 08:53:20,766 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-1-shard-people-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:53:20,766 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement-notifier#1719464469] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:20,769 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-config_testModuleShardLeaderMovement: journal open: applyTo=0 08:53:20,771 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:53:20,773 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:53:20,773 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testModuleShardLeaderMovement is initialized 08:53:20,773 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:20,774 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:20,774 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 08:53:20,774 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:20,778 AM [cluster-test-shard-dispatcher-10] [INFO] Candidate - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 08:53:20,778 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:20,778 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:20,778 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:20,778 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:53:20,778 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:20,778 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:53:20,778 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 08:53:20,778 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:53:20,778 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 08:53:20,779 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3ed58d6b 08:53:20,779 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 08:53:20,779 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:53:20,779 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:53:20,779 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:20,779 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testModuleShardLeaderMovement is initialized 08:53:20,780 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:20,780 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from null to Follower 08:53:20,785 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate): Starting new election term 1 08:53:20,785 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:20,785 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:20,785 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:20,785 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:53:20,785 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:20,785 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:53:20,785 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: became leader with frontend state for [] 08:53:20,785 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:53:20,786 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 08:53:20,786 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@8099087 08:53:20,786 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testModuleShardLeaderMovement from Candidate to Leader 08:53:20,786 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:20,799 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-940410098] 08:53:20,799 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:20,799 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223] 08:53:20,800 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:20,803 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:20,807 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 08:53:20,811 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:53:20,811 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:53:20,812 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1656951693] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:20,813 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:53:20,813 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testModuleShardLeaderMovement 08:53:20,814 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testModuleShardLeaderMovement 08:53:20,814 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:20,814 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 08:53:20,814 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 08:53:20,840 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 08:53:20,842 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@41e8c446} 08:53:20,842 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 08:53:20,843 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:20,843 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testModuleShardLeaderMovement 08:53:20,844 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 08:53:20,844 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 08:53:20,844 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement on shard member-1-shard-cars-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#1298776529] 08:53:20,844 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testModuleShardLeaderMovement 08:53:20,845 AM [cluster-test-shard-dispatcher-10] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 08:53:20,845 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 08:53:20,845 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 08:53:20,845 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testModuleShardLeaderMovement on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement on shard member-1-shard-people-oper_testModuleShardLeaderMovement whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#1796683317] 08:53:20,845 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Shard created, persistent : true 08:53:20,846 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#1298776529] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:20,846 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-people-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:53:20,846 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#1341114967 created and ready for shard:member-1-shard-people-oper_testModuleShardLeaderMovement 08:53:20,846 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement#1796683317] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:20,846 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#-1858115264 created and ready for shard:member-1-shard-cars-oper_testModuleShardLeaderMovement 08:53:20,847 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement-notifier#1341114967] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:20,847 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActor - member-1-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:53:20,847 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement-notifier#-1858115264] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:20,848 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testModuleShardLeaderMovement: journal open: applyTo=0 08:53:20,849 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-people-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:20,850 AM [cluster-test-shard-dispatcher-10] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 08:53:20,851 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:20,860 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-people-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:53:20,861 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:20,861 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:20,861 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:53:20,861 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testModuleShardLeaderMovement is initialized 08:53:20,861 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from null to Follower 08:53:20,864 AM [cluster-test-shard-dispatcher-10] [INFO] PekkoRecovery - member-1-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:53:20,865 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:53:20,865 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testModuleShardLeaderMovement is initialized 08:53:20,865 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:20,865 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:20,865 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 08:53:20,868 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 08:53:20,868 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:20,868 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:20,868 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:20,868 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:53:20,868 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:20,868 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:53:20,868 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 08:53:20,868 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:53:20,868 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 08:53:20,868 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@59df3313 08:53:20,868 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testModuleShardLeaderMovement from Candidate to Leader 08:53:20,869 AM [cluster-test-shard-dispatcher-10] [INFO] Candidate - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate): Starting new election term 1 08:53:20,869 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:20,869 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:20,869 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:20,869 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:53:20,869 AM [cluster-test-shard-dispatcher-10] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:20,870 AM [cluster-test-shard-dispatcher-10] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:53:20,870 AM [cluster-test-shard-dispatcher-10] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: became leader with frontend state for [] 08:53:20,872 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:53:20,872 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 08:53:20,872 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ae64c16 08:53:20,872 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Candidate to Leader 08:53:20,872 AM [cluster-test-shard-dispatcher-10] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:20,893 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-1656951693] 08:53:20,893 AM [cluster-test-shard-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:53:20,893 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#1298776529] 08:53:20,907 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:53:20,928 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [7713734278875405916] 08:53:20,929 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 08:53:20,932 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:20,932 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:53:20,937 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:20,937 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:20,948 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:20,950 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:20,962 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 08:53:20,977 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-133867539] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:20,978 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:53:20,978 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:53:20,982 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:53:20,985 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 08:53:20,987 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@60aa31f6} 08:53:20,987 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 08:53:20,988 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:20,990 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:20,991 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:20,994 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 08:53:20,996 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#332881165] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:20,996 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:53:20,996 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:53:20,997 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:53:21,9 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 08:53:21,10 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@6e6dc393} 08:53:21,10 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 08:53:21,15 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:21,45 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:53:21,52 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:53:21,72 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:53:21,74 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:21,74 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:21,74 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:21,74 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:21,81 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [6968773063366869824] 08:53:21,82 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 08:53:21,85 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:21,85 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:53:21,88 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:21,88 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:21,98 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:21,99 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:21,102 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testModuleShardLeaderMovement 08:53:21,109 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#240760262] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:21,112 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:53:21,112 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:53:21,118 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:53:21,125 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 08:53:21,128 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} started 08:53:21,129 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:21,131 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:21,132 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@3fc009d3} 08:53:21,133 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:21,133 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testModuleShardLeaderMovement 08:53:21,135 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:53:21,135 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:53:21,136 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-108788309] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:21,139 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:53:21,145 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:21,145 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:21,146 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:53:21,153 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:53:21,153 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:21,153 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:21,160 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testModuleShardLeaderMovement: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 08:53:21,161 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@679d784c} 08:53:21,161 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} started 08:53:21,161 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:21,716 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:53:21,717 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:53:21,717 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:21,717 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement 08:53:21,717 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 08:53:21,717 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement 08:53:21,717 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 08:53:21,717 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:21,717 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:21,717 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement 08:53:21,717 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement 08:53:21,717 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement 08:53:21,717 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-people-oper_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-people-oper_testModuleShardLeaderMovement 08:53:21,717 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:21,718 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:21,718 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement 08:53:21,718 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 08:53:21,718 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement 08:53:21,718 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 08:53:21,718 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:21,718 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:21,718 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement 08:53:21,718 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement 08:53:21,718 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement 08:53:21,718 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testModuleShardLeaderMovement with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-people-config_testModuleShardLeaderMovement to pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-people-config_testModuleShardLeaderMovement 08:53:21,719 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:21,727 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:21,727 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:21,727 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:21,727 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:21,727 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:21,727 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:21,727 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:21,727 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:21,973 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:21,973 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:21,973 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:21,973 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:21,973 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:21,973 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:21,973 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:21,973 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,67 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:53:22,68 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 08:53:22,69 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:22,69 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement] 08:53:22,74 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:53:22,74 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223, primaryVersion=13] 08:53:22,79 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223 for shard member-2-shard-cars-config_testModuleShardLeaderMovement 08:53:22,80 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 08:53:22,81 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:53:22,82 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testModuleShardLeaderMovement 08:53:22,82 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#-1287037937 created and ready for shard:member-2-shard-cars-config_testModuleShardLeaderMovement 08:53:22,82 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-791185503] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,82 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-notifier#-1287037937] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,83 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:53:22,83 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 08:53:22,83 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:22,84 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-2-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:22,91 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-2-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:53:22,92 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2026-04-10T08:53:22.083253644Z 08:53:22,92 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:53:22,92 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testModuleShardLeaderMovement is initialized 08:53:22,92 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:22,92 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,93 AM [cluster-test-shard-dispatcher-16] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:53:22,93 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 08:53:22,98 AM [cluster-test-shard-dispatcher-16] [INFO] Follower - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 08:53:22,99 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:53:22,99 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:22,100 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 08:53:22,100 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2f50c9a8 08:53:22,101 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from null to Follower 08:53:22,101 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,105 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2026-04-10T08:53:22.099712751Z 08:53:22,105 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 08:53:22,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:22,105 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#925937901] with name member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 08:53:22,106 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 724.0 μs 08:53:22,106 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:53:22,106 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#925937901] 08:53:22,107 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 08:53:22,107 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 08:53:22,107 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 98.31 μs 08:53:22,107 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#925937901] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,109 AM [cluster-test-shard-dispatcher-29] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:53:22,113 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223 08:53:22,113 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 08:53:22,114 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testModuleShardLeaderMovement 08:53:22,114 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 08:53:22,150 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:22,152 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 08:53:22,152 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,152 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:53:22,154 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4dec3487 08:53:22,154 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-133867539] 08:53:22,154 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,154 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:53:22,154 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:22,154 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-791185503] 08:53:22,154 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-791185503] 08:53:22,155 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#332881165] 08:53:22,155 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:53:22,155 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 08:53:22,156 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:53:22,156 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 08:53:22,160 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:22,160 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement] 08:53:22,162 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:53:22,162 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#1298776529, primaryVersion=13] 08:53:22,167 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 08:53:22,168 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:53:22,168 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#1298776529 for shard member-2-shard-cars-oper_testModuleShardLeaderMovement 08:53:22,169 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#926549785] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,169 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#1235407285] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,170 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 08:53:22,170 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-notifier#1235407285 created and ready for shard:member-2-shard-cars-oper_testModuleShardLeaderMovement 08:53:22,171 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testModuleShardLeaderMovement 08:53:22,171 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:53:22,172 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:22,172 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:22,178 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2026-04-10T08:53:22.171992995Z 08:53:22,179 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:53:22,180 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:22,180 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:53:22,180 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,180 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testModuleShardLeaderMovement is initialized 08:53:22,180 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:53:22,180 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 08:53:22,189 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 08:53:22,190 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:22,190 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 08:53:22,190 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 08:53:22,190 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7042b4b7 08:53:22,190 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 08:53:22,190 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:22,198 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2026-04-10T08:53:22.190656295Z 08:53:22,198 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 08:53:22,198 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:22,199 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-379558956] with name member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 08:53:22,199 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-379558956] 08:53:22,199 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 556.5 μs 08:53:22,199 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 08:53:22,200 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-379558956] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,200 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 08:53:22,201 AM [cluster-test-shard-dispatcher-29] [INFO] AbstractLeader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:53:22,201 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-2-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 141.3 μs 08:53:22,205 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@19fb93af 08:53:22,206 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:22,206 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#1298776529 08:53:22,206 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 08:53:22,206 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testModuleShardLeaderMovement 08:53:22,206 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 08:53:22,212 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:22,213 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:53:22,213 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 08:53:22,213 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#332881165] 08:53:22,213 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 08:53:22,213 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,213 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:53:22,213 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#926549785] 08:53:22,213 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-2-shard-cars-oper_testModuleShardLeaderMovement#926549785] 08:53:22,214 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:53:22,214 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 08:53:22,214 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:22,214 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement] 08:53:22,217 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:53:22,217 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223, primaryVersion=13] 08:53:22,220 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223 for shard member-3-shard-cars-config_testModuleShardLeaderMovement 08:53:22,221 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Shard created, persistent : true 08:53:22,221 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,222 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#790619994 created and ready for shard:member-3-shard-cars-config_testModuleShardLeaderMovement 08:53:22,222 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:53:22,222 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-notifier#790619994] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,223 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testModuleShardLeaderMovement: journal open: applyTo=0 08:53:22,224 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testModuleShardLeaderMovement 08:53:22,225 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 08:53:22,225 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:22,225 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:22,234 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2026-04-10T08:53:22.225292372Z 08:53:22,234 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:53:22,235 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:53:22,235 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testModuleShardLeaderMovement is initialized 08:53:22,235 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:22,235 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,236 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from null to Follower 08:53:22,236 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]}}" message is greater than follower's term 0 - updating term 08:53:22,271 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-config_testModuleShardLeaderMovement 08:53:22,272 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:53:22,272 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:22,272 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement 08:53:22,272 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@722fdaae 08:53:22,272 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from null to Follower 08:53:22,273 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,285 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-3-shard-cars-config_testModuleShardLeaderMovement: snapshot is durable as of 2026-04-10T08:53:22.272652832Z 08:53:22,285 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Applying snapshot 08:53:22,285 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:22,285 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#338433231] with name member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher 08:53:22,286 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#338433231] 08:53:22,286 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: state snapshot applied in 743.6 μs 08:53:22,286 AM [cluster-test-shard-dispatcher-12] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Done applying snapshot 08:53:22,286 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testModuleShardLeaderMovement: publishChanges: rootPath: / 08:53:22,286 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher: 83.32 μs 08:53:22,286 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement-DTCL-publisher#338433231] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,288 AM [cluster-test-shard-dispatcher-29] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 08:53:22,291 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223 08:53:22,292 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 08:53:22,293 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testModuleShardLeaderMovement 08:53:22,293 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 08:53:22,293 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:22,296 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:53:22,296 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:53:22,298 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 08:53:22,298 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,300 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@d39f1a9 08:53:22,300 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:53:22,300 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,300 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:53:22,300 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#240760262] 08:53:22,301 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:22,301 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] 08:53:22,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] 08:53:22,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-108788309] 08:53:22,301 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:53:22,301 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 08:53:22,301 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:53:22,301 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 08:53:22,302 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:22,302 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement] 08:53:22,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:53:22,304 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#1298776529, primaryVersion=13] 08:53:22,306 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#1298776529 for shard member-3-shard-cars-oper_testModuleShardLeaderMovement 08:53:22,307 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Shard created, persistent : true 08:53:22,307 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Starting recovery with journal batch size 1 08:53:22,309 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testModuleShardLeaderMovement: journal open: applyTo=0 08:53:22,309 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#2112679041] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,309 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testModuleShardLeaderMovement 08:53:22,309 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#-871013084 created and ready for shard:member-3-shard-cars-oper_testModuleShardLeaderMovement 08:53:22,309 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-notifier#-871013084] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,310 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 08:53:22,310 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:22,311 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testModuleShardLeaderMovement: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:22,313 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-1-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2026-04-10T08:53:22.310267648Z 08:53:22,315 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testModuleShardLeaderMovement: Local TermInfo store seeded with TermInfo{term=0} 08:53:22,315 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:53:22,315 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:22,315 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testModuleShardLeaderMovement is initialized 08:53:22,315 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,315 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , received role change from null to Follower 08:53:22,315 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testModuleShardLeaderMovement, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]}}" message is greater than follower's term 0 - updating term 08:53:22,318 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-cars-oper_testModuleShardLeaderMovement (Follower): Snapshot received from leader: member-1-shard-cars-oper_testModuleShardLeaderMovement 08:53:22,318 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:22,318 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testModuleShardLeaderMovement , registered listener pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement 08:53:22,318 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done false 08:53:22,318 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c722b34 08:53:22,318 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testModuleShardLeaderMovement from null to Follower 08:53:22,318 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:22,326 AM [cluster-test-shard-dispatcher-29] [INFO] SnapshotManager - member-3-shard-cars-oper_testModuleShardLeaderMovement: snapshot is durable as of 2026-04-10T08:53:22.318742346Z 08:53:22,326 AM [cluster-test-shard-dispatcher-29] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testModuleShardLeaderMovement: Applying snapshot 08:53:22,326 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testModuleShardLeaderMovement: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:22,327 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testModuleShardLeaderMovement: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1299918679] with name member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher 08:53:22,327 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1299918679] 08:53:22,327 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testModuleShardLeaderMovement: state snapshot applied in 536.3 μs 08:53:22,327 AM [cluster-test-shard-dispatcher-29] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testModuleShardLeaderMovement: Done applying snapshot 08:53:22,327 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testModuleShardLeaderMovement: publishChanges: rootPath: / 08:53:22,328 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher#-1299918679] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,327 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testModuleShardLeaderMovement: Elapsed time for generation of change events for member-3-shard-cars-oper_testModuleShardLeaderMovement-DTCL-publisher: 143.0 μs 08:53:22,329 AM [cluster-test-shard-dispatcher-29] [INFO] AbstractLeader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testModuleShardLeaderMovement (last chunk 1) - matchIndex set to 0, nextIndex set to 1 08:53:22,332 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testModuleShardLeaderMovement} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testModuleShardLeaderMovement/member-1-shard-cars-oper_testModuleShardLeaderMovement#1298776529 08:53:22,333 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 08:53:22,333 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testModuleShardLeaderMovement 08:53:22,333 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 08:53:22,335 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-2-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testModuleShardLeaderMovement, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:22,340 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done false 08:53:22,340 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testModuleShardLeaderMovement status sync done true 08:53:22,341 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : true 08:53:22,341 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,342 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@508460be 08:53:22,342 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:53:22,342 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testModuleShardLeaderMovement is ready 08:53:22,342 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testModuleShardLeaderMovement status sync done true 08:53:22,342 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement#-108788309] 08:53:22,342 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:53:22,342 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#2112679041] 08:53:22,343 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testModuleShardLeaderMovement/member-3-shard-cars-oper_testModuleShardLeaderMovement#2112679041] 08:53:22,343 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-940410098] 08:53:22,343 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:22,343 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223] 08:53:22,343 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223] 08:53:22,344 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-133867539] 08:53:22,344 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:22,344 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-791185503] 08:53:22,344 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-791185503] 08:53:22,344 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#240760262] 08:53:22,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:22,344 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] 08:53:22,344 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] 08:53:22,346 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-1 for shard ShardName{value=cars}, datastoreType Config 08:53:22,346 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-940410098] 08:53:22,346 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:22,348 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223] 08:53:22,348 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223] 08:53:22,348 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 08:53:22,348 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 08:53:22,350 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-940410098] 08:53:22,350 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:22,350 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223] 08:53:22,350 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223] 08:53:22,362 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-133867539] 08:53:22,363 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:22,363 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-791185503] 08:53:22,363 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-791185503] 08:53:22,363 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#240760262] 08:53:22,363 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:22,363 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] 08:53:22,364 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] 08:53:22,364 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-2 for shard ShardName{value=cars}, datastoreType Config 08:53:22,364 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-133867539] 08:53:22,364 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:22,364 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-791185503] 08:53:22,364 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-791185503] 08:53:22,364 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 08:53:22,374 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3713c0ae 08:53:22,374 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7b3210f9 08:53:22,374 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7b3210f9 08:53:22,374 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 08:53:22,375 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5ed79fd5 08:53:22,376 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7cb768c6 08:53:22,376 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:53:22,377 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 08:53:22,378 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:53:22,425 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 2 08:53:22,425 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:53:22,425 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:22,425 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,426 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:53:22,426 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1e85bad4 08:53:22,426 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 08:53:22,427 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 2 in "RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:53:22,427 AM [cluster-test-shard-dispatcher-29] [INFO] AbstractLeader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 2 in "RequestVote{term=2, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 1 - switching to Follower 08:53:22,442 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 2 08:53:22,443 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:22,443 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,443 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 08:53:22,443 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5dda1e1e 08:53:22,443 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 2 08:53:22,443 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 08:53:22,443 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:53:22,443 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 08:53:22,443 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:53:22,443 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@230af0aa 08:53:22,443 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 08:53:22,443 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,445 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 08:53:22,445 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 71.18 ms 08:53:22,445 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1913208a 08:53:22,445 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,446 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:53:22,447 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:53:22,451 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 08:53:22,451 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-133867539] 08:53:22,451 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:22,451 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-791185503] 08:53:22,452 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-791185503] 08:53:22,452 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:53:22,452 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1ca1991f 08:53:22,452 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,453 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-940410098] 08:53:22,453 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:22,453 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223] 08:53:22,453 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223] 08:53:22,453 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#240760262] 08:53:22,453 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:22,454 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] 08:53:22,454 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] 08:53:22,454 AM [main] [INFO] ClusterAdminRpcService - Moving leader to local node member-3 for shard ShardName{value=cars}, datastoreType Config 08:53:22,454 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#240760262] 08:53:22,454 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:22,454 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] 08:53:22,454 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.MakeLeaderLocal to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] 08:53:22,454 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onMakeLeaderLocal received 08:53:22,455 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:53:22,456 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6a46da9 08:53:22,457 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@738caaab 08:53:22,457 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@738caaab 08:53:22,457 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 08:53:22,458 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1ede0d1a 08:53:22,458 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@51c72a34 08:53:22,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 08:53:22,490 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 3 08:53:22,491 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 3 08:53:22,491 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:22,491 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,491 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:53:22,491 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@22b52122 08:53:22,491 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 08:53:22,493 AM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 3 in "RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 2 - switching to Follower 08:53:22,493 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 3 in "RequestVote{term=3, candidateId=member-3-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 2 - updating term 08:53:22,525 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:22,525 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:53:22,525 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 08:53:22,526 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:53:22,526 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@407e3b96 08:53:22,526 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 08:53:22,526 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,527 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:53:22,527 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@37e1b8df 08:53:22,527 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,530 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 3 08:53:22,530 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:22,530 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,530 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 08:53:22,530 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2aa2fe4e 08:53:22,530 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 08:53:22,530 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:53:22,531 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-3-shard-cars-config_testModuleShardLeaderMovement 08:53:22,531 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@681947be 08:53:22,531 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,531 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-3-shard-cars-config_testModuleShardLeaderMovement in 74.43 ms 08:53:22,533 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:53:22,533 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:53:22,534 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ClusterAdminRpcService - Leadership transfer complete 08:53:22,534 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#240760262] 08:53:22,534 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:22,534 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] 08:53:22,534 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement#576663021] 08:53:22,535 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 08:53:22,535 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testModuleShardLeaderMovement 08:53:22,536 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 08:53:22,536 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@16db9137 08:53:22,536 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@50872be7 08:53:22,536 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-3-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@50872be7 08:53:22,536 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 08:53:22,538 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@62da3edd 08:53:22,538 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3ac66ad3 08:53:22,539 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 08:53:22,544 AM [cluster-test-shard-dispatcher-28] [INFO] Candidate - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 4 08:53:22,545 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 4 08:53:22,545 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:22,545 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,545 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:53:22,545 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6c1ddcf6 08:53:22,545 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 08:53:22,546 AM [cluster-test-shard-dispatcher-16] [INFO] Follower - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than follower's term 3 - updating term 08:53:22,546 AM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 3 - switching to Follower 08:53:22,566 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-3-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 4 08:53:22,566 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:22,566 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,566 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 08:53:22,566 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@38f93d49 08:53:22,566 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 08:53:22,566 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 08:53:22,567 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:53:22,567 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 08:53:22,567 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:53:22,567 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@682255ad 08:53:22,567 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 08:53:22,567 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,568 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:53:22,568 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2c785adf 08:53:22,569 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,569 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-2-shard-cars-config_testModuleShardLeaderMovement 08:53:22,569 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-2-shard-cars-config_testModuleShardLeaderMovement in 33.58 ms 08:53:22,570 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@33cea745 08:53:22,570 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,570 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:53:22,570 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 08:53:22,571 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1 Received follower initial sync status for member-1-shard-cars-config_testModuleShardLeaderMovement status sync done true 08:53:22,572 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testModuleShardLeaderMovement: journal closed 08:53:22,572 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:53:22,572 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:53:22,572 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 08:53:22,573 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement#-791185503] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-3-shard-cars-config_testModuleShardLeaderMovement] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,573 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#240760262] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#240760262] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,573 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testModuleShardLeaderMovement 08:53:22,573 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 08:53:22,574 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 08:53:22,574 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testModuleShardLeaderMovement: journal closed 08:53:22,575 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:53:22,575 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:53:22,577 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:22,577 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:22,583 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:22,583 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:22,583 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:22,584 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:22,585 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:53:22,607 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 08:53:22,608 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testModuleShardLeaderMovement 08:53:22,608 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 08:53:22,608 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2d20d6cd 08:53:22,609 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5aefd7e9 08:53:22,609 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-2-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5aefd7e9 08:53:22,609 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 08:53:22,612 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5d37070 08:53:22,614 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader transfer complete - waiting for new leader 08:53:22,641 AM [cluster-test-shard-dispatcher-29] [INFO] Candidate - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate): Starting new election term 5 08:53:22,641 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Follower) :- Switching from behavior Follower to Candidate, election term: 5 08:53:22,641 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:22,641 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,642 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Follower to Candidate 08:53:22,642 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@53bca62a 08:53:22,642 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Follower to Candidate 08:53:22,643 AM [cluster-test-shard-dispatcher-16] [INFO] AbstractLeader - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader): Term 5 in "RequestVote{term=5, candidateId=member-1-shard-cars-config_testModuleShardLeaderMovement, lastLogIndex=1, lastLogTerm=1}" message is greater than leader's term 4 - switching to Follower 08:53:22,667 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-2-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to Follower, election term: 5 08:53:22,667 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: false, hasLeader : false 08:53:22,667 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:22,667 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to Follower 08:53:22,667 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7f033038 08:53:22,667 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testModuleShardLeaderMovement from Leader to Follower 08:53:22,668 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 08:53:22,668 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:53:22,668 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: became leader with frontend state for [] 08:53:22,669 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Candidate to Leader 08:53:22,669 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@38e2d17c 08:53:22,669 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Candidate to Leader 08:53:22,669 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,670 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: leader changed to member-1-shard-cars-config_testModuleShardLeaderMovement 08:53:22,670 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testModuleShardLeaderMovement: Successfully transferred leadership to member-1-shard-cars-config_testModuleShardLeaderMovement in 61.42 ms 08:53:22,670 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1e3f2962 08:53:22,670 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testModuleShardLeaderMovement is ready 08:53:22,670 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testModuleShardLeaderMovement status sync done false 08:53:22,671 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-2-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 08:53:22,672 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testModuleShardLeaderMovement: journal closed 08:53:22,672 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:53:22,673 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:53:22,674 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testModuleShardLeaderMovement/member-1-shard-cars-config_testModuleShardLeaderMovement#1930406223] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement/member-2-shard-cars-config_testModuleShardLeaderMovement] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,674 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 08:53:22,675 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testModuleShardLeaderMovement 08:53:22,675 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 08:53:22,675 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-133867539] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testModuleShardLeaderMovement#-133867539] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:22,676 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 08:53:22,676 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testModuleShardLeaderMovement: journal closed 08:53:22,676 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:53:22,677 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:53:22,678 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:22,678 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:22,735 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:53:23,595 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [6968773063366869824] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:53:23,696 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:23,696 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [7713734278875405916] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:53:23,696 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:23,698 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:53:23.701] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:23,711 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testModuleShardLeaderMovement, generation=0} 08:53:23,712 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testModuleShardLeaderMovement 08:53:23,712 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testModuleShardLeaderMovement 08:53:23,712 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:53:23,713 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5622da1d 08:53:23,713 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@4700ddf4 08:53:23,713 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-people-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@4700ddf4 08:53:23,713 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3ad11a71 08:53:23,713 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3ad11a71 08:53:23,713 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 08:53:23,714 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-config_testModuleShardLeaderMovement: Stopping Shard 08:53:23,715 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-config_testModuleShardLeaderMovement: journal closed 08:53:27,836 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 08:53:27,836 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 08:53:27,837 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:53:27,837 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:27,837 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:27,838 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:27,838 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:30,34 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:31,312 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:53:32,905 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: leader transfer aborted 08:53:32,905 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testModuleShardLeaderMovement: Failed to transfer leadership in 9.192 s 08:53:32,905 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: In unpauseLeader 08:53:32,906 AM [cluster-test-shard-dispatcher-17] [WARN] Leader - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 08:53:32,906 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-config_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 5 08:53:32,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:53:32,906 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 08:53:32,907 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testModuleShardLeaderMovement from Leader to IsolatedLeader 08:53:32,907 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testModuleShardLeaderMovement: Stopping Shard 08:53:32,908 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testModuleShardLeaderMovement: journal closed 08:53:32,908 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:53:32,908 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:53:32,909 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testModuleShardLeaderMovement, generation=0} 08:53:32,910 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testModuleShardLeaderMovement 08:53:32,910 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testModuleShardLeaderMovement 08:53:32,910 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:53:32,910 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@48ec7059 08:53:32,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6408a661 08:53:32,910 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@678d7401 08:53:32,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6408a661 08:53:32,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: pauseLeader successfully completed - doing transfer 08:53:32,910 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-oper_testModuleShardLeaderMovement: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@48ec7059 08:53:32,911 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-people-oper_testModuleShardLeaderMovement: Stopping Shard 08:53:32,911 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testModuleShardLeaderMovement: journal closed 08:53:36,245 AM [cluster-test-shard-dispatcher-29] [WARN] Leader - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 08:53:36,245 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testModuleShardLeaderMovement (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:53:36,245 AM [cluster-test-shard-dispatcher-29] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: leader transfer aborted 08:53:36,246 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testModuleShardLeaderMovement: Failed to transfer leadership in 3.336 s 08:53:36,246 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: In unpauseLeader 08:53:36,246 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testModuleShardLeaderMovement: onLeadershipChange, isLeader: true, hasLeader : true 08:53:36,246 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testModuleShardLeaderMovement , received role change from Leader to IsolatedLeader 08:53:36,246 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testModuleShardLeaderMovement from Leader to IsolatedLeader 08:53:36,246 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-1-shard-cars-oper_testModuleShardLeaderMovement: Stopping Shard 08:53:36,247 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testModuleShardLeaderMovement: journal closed 08:53:36,247 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:53:36,248 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:53:36,250 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:36,250 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:36,254 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:53:36.256] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:36,277 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:53:36,290 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [2077208803373086085] 08:53:36,290 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:53:36,291 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:53:36,291 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:53:36,292 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:36,292 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:36,307 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:53:36,307 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:53:36,307 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:53:36,309 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:36,311 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:36,313 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 08:53:36,323 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-640495729] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,324 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:53:36,324 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:53:36,330 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:53:36,330 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardReplica 08:53:36,330 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardReplica 08:53:36,331 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:36,331 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:53:36,331 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:53:36,333 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:53:36,333 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@5a3090f0} 08:53:36,334 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} started 08:53:36,334 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:36,336 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:36,337 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:36,337 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardReplica 08:53:36,345 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 08:53:36,351 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:53:36,351 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:53:36,351 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica on shard member-1-shard-cars-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2067251677] 08:53:36,351 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardReplica 08:53:36,352 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 08:53:36,352 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:53:36,352 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:53:36,352 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica on shard member-1-shard-people-config_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#236194441] 08:53:36,352 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#1232642609] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,353 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 08:53:36,354 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 08:53:36,354 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2067251677] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,354 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#1399526828 created and ready for shard:member-1-shard-cars-config_testRemoveShardReplica 08:53:36,354 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#215118366 created and ready for shard:member-1-shard-people-config_testRemoveShardReplica 08:53:36,355 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 08:53:36,360 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:53:36,360 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:53:36,361 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:53:36,361 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardReplica 08:53:36,361 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardReplica 08:53:36,362 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:36,362 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:36,362 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:36,362 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@71f25f1e} 08:53:36,362 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:36,362 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardReplica 08:53:36,365 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:36,367 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 08:53:36,368 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 08:53:36,368 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:36,369 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 08:53:36,370 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#-690257705 created and ready for shard:member-1-shard-cars-oper_testRemoveShardReplica 08:53:36,371 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:36,372 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica#236194441] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,372 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica-notifier#1399526828] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,372 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica-notifier#215118366] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,372 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#2063565002] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,372 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica-notifier#-690257705] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,372 AM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:36,373 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:36,377 AM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:36,378 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,378 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,378 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 08:53:36,378 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:53:36,378 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardReplica is initialized 08:53:36,378 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:36,378 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:36,378 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-1-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica#2063565002] 08:53:36,378 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardReplica 08:53:36,379 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:36,379 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:36,379 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-1-shard-people-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#1886485660] 08:53:36,379 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:53:36,379 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardReplica is initialized 08:53:36,379 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 08:53:36,380 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:36,381 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 08:53:36,381 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:53:36,381 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#-1934921387 created and ready for shard:member-1-shard-people-oper_testRemoveShardReplica 08:53:36,381 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica#1886485660] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,382 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from null to Follower 08:53:36,382 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica-notifier#-1934921387] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,386 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:36,386 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,386 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,387 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 08:53:36,387 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:53:36,387 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from null to Follower 08:53:36,388 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:36,388 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:53:36,389 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:36,389 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,389 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,389 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:53:36,389 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardReplica is initialized 08:53:36,390 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 08:53:36,392 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 08:53:36,392 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:53:36,392 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from null to Follower 08:53:36,393 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:36,395 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:36,396 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:53:36,396 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardReplica is initialized 08:53:36,396 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,396 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,396 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from null to Follower 08:53:36,396 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:53:36,396 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from null to Follower 08:53:36,409 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-2981246241505860661] 08:53:36,410 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 08:53:36,412 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:36,413 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:53:36,414 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:36,414 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:36,428 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:36,438 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:36,440 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 08:53:36,451 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:53:36,451 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:53:36,451 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#2060258265] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,458 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:53:36,458 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 08:53:36,458 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 08:53:36,460 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:53:36,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@4826a1e} 08:53:36,461 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} started 08:53:36,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:36,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 08:53:36,461 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:36,462 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 08:53:36,463 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:36,463 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 08:53:36,463 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:36,464 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1173054730] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,464 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#773328236 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 08:53:36,464 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#773328236] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,464 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 08:53:36,465 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#636245696] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,465 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-1921421647 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 08:53:36,465 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#-1921421647] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,466 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 08:53:36,467 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 08:53:36,468 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:53:36,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:53:36,469 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:36,470 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-2135792134] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,471 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:36,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,472 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:53:36,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,472 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 08:53:36,472 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 08:53:36,472 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:53:36,476 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 08:53:36,477 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:36,479 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 08:53:36,481 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:53:36,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 08:53:36,483 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:36,484 AM [cluster-test-pekko.actor.default-dispatcher-29] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 08:53:36,484 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 08:53:36,485 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@1ce4f04a} 08:53:36,485 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 08:53:36,486 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:36,486 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 08:53:36,486 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 08:53:36,487 AM [cluster-test-pekko.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#-508467729] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,487 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 08:53:36,488 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#-521327732 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 08:53:36,488 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#-1471502024] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,488 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#-521327732] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,488 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 08:53:36,488 AM [cluster-test-shard-dispatcher-32] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:36,490 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-96367118 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 08:53:36,490 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#-96367118] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,490 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:36,491 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:53:36,491 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 08:53:36,491 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,491 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,491 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 08:53:36,491 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:53:36,492 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 08:53:36,492 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 08:53:36,495 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:36,495 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:36,497 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:36,497 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,497 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,497 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:53:36,497 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 08:53:36,498 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 08:53:36,498 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 08:53:36,499 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:53:36,499 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 08:53:36,499 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:36,501 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:36,502 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,502 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:53:36,504 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 08:53:36,504 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,504 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 08:53:36,504 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:53:36,504 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 08:53:36,505 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:53:36,509 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:53:36,509 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:36,509 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:36,509 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:53:36,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:53:36,509 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:53:36,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:53:36,510 AM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:53:36,510 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:53:36,510 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:36,512 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:53:36,513 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:36,513 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:53:36,513 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:36,513 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:36,514 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:53:36,523 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [4153471639277276822] 08:53:36,524 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 08:53:36,526 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:36,526 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:53:36,532 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:36,533 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:36,541 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:36,542 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:36,548 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 08:53:36,555 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:53:36,555 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:53:36,556 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1758622112] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,559 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:53:36,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardReplica 08:53:36,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardReplica 08:53:36,562 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:53:36,563 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@473cfea1} 08:53:36,569 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} started 08:53:36,570 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:36,570 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:36,570 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardReplica 08:53:36,571 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:36,573 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 08:53:36,577 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:53:36,579 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardReplica 08:53:36,580 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1319609591] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,580 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-412253607] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,580 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 08:53:36,581 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:36,582 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 08:53:36,583 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:53:36,583 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#1060791990] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,585 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 08:53:36,585 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica#-336956180] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,585 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:36,585 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#2022242053 created and ready for shard:member-3-shard-people-config_testRemoveShardReplica 08:53:36,586 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica-notifier#2022242053] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,586 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:36,587 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 08:53:36,588 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica-notifier#1060791990 created and ready for shard:member-3-shard-cars-config_testRemoveShardReplica 08:53:36,589 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:53:36,589 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:53:36,589 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:36,590 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:53:36,590 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:36,590 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardReplica 08:53:36,591 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:36,592 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,592 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,592 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:53:36,592 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardReplica is initialized 08:53:36,592 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,592 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,592 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , received role change from null to Follower 08:53:36,592 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 08:53:36,592 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:53:36,593 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardReplica 08:53:36,595 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:36,595 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:36,595 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:36,597 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:53:36,597 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardReplica is initialized 08:53:36,597 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardReplica from null to Follower 08:53:36,597 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:36,597 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:53:36,598 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:53:36,598 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:53:36,598 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:53:36,598 AM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:53:36,598 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:53:36,598 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:53:36,598 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardReplica from null to Follower 08:53:36,600 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 08:53:36,601 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@6a3187ac} 08:53:36,601 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} started 08:53:36,601 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:53:36,602 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:36,602 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardReplica 08:53:36,602 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:36,602 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:53:36,602 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica on shard member-3-shard-cars-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#823695815] 08:53:36,602 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardReplica 08:53:36,603 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:36,603 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:53:36,603 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica on shard member-3-shard-people-oper_testRemoveShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#-2064360491] 08:53:36,603 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica#823695815] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,603 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 08:53:36,604 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 08:53:36,605 AM [cluster-test-shard-dispatcher-34] [INFO] RaftActor - member-3-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:36,607 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica#-2064360491] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,607 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#339028203 created and ready for shard:member-3-shard-people-oper_testRemoveShardReplica 08:53:36,607 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica-notifier#339028203] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,607 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#1597261695 created and ready for shard:member-3-shard-cars-oper_testRemoveShardReplica 08:53:36,608 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica-notifier#1597261695] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:36,608 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:36,609 AM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 08:53:36,610 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 08:53:36,610 AM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:36,611 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:36,614 AM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:36,614 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:53:36,615 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardReplica is initialized 08:53:36,615 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,615 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,615 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 08:53:36,615 AM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:36,615 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:53:36,615 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardReplica from null to Follower 08:53:36,615 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:36,616 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:53:36,616 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardReplica is initialized 08:53:36,616 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,616 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,616 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 08:53:36,616 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:36,616 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:53:36,617 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardReplica from null to Follower 08:53:36,738 AM [cluster-test-shard-dispatcher-25] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 1 08:53:36,738 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:36,738 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,738 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,739 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Follower to Candidate 08:53:36,739 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Follower to Candidate 08:53:36,786 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 1 08:53:36,786 AM [cluster-test-shard-dispatcher-25] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 1 08:53:36,786 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:36,786 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,787 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:36,787 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,787 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,787 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,787 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Follower to Candidate 08:53:36,787 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Follower to Candidate 08:53:36,787 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Follower to Candidate 08:53:36,787 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Follower to Candidate 08:53:36,787 AM [cluster-test-shard-dispatcher-26] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 1 08:53:36,787 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:36,788 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:36,788 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:36,788 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Follower to Candidate 08:53:36,788 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Follower to Candidate 08:53:37,126 AM [cluster-test-shard-dispatcher-26] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 2 08:53:37,126 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 2 08:53:37,146 AM [cluster-test-shard-dispatcher-26] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 2 08:53:37,195 AM [cluster-test-shard-dispatcher-26] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 2 08:53:37,304 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:53:37,304 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:53:37,305 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:37,305 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:37,305 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:53:37,305 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:37,306 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardReplica: Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:37,306 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:53:37,306 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:53:37,314 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:37,314 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:53:37,314 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:53:37,314 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:53:37,314 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:37,315 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:37,315 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:37,315 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:53:37,315 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:37,450 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:37,450 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:37,450 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:53:37,450 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardReplica: Peer address for peer member-2-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:53:37,451 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:37,451 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:53:37,451 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:53:37,451 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:53:37,451 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:53:37,451 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:53:37,451 AM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:53:37,476 AM [cluster-test-shard-dispatcher-25] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardReplica (Candidate): Starting new election term 3 08:53:37,479 AM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-3-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:37,480 AM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:37,487 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:37,487 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:37,487 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: became leader with frontend state for [] 08:53:37,488 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Candidate to Leader 08:53:37,488 AM [cluster-test-shard-dispatcher-25] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardReplica (Candidate): Starting new election term 3 08:53:37,488 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1dc82cf6 08:53:37,488 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Candidate to Leader 08:53:37,490 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done false 08:53:37,490 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-3-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:37,490 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 08:53:37,491 AM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:37,491 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@78930099 08:53:37,491 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@70988ba8 08:53:37,496 AM [cluster-test-shard-dispatcher-26] [INFO] Candidate - member-1-shard-people-config_testRemoveShardReplica (Candidate): Starting new election term 3 08:53:37,497 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:37,497 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:37,497 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: became leader with frontend state for [] 08:53:37,497 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@579ed7aa 08:53:37,497 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Candidate to Leader 08:53:37,497 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Candidate to Leader 08:53:37,497 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:53:37,498 AM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:37,498 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-3-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:37,498 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done false 08:53:37,499 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 08:53:37,499 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1767d41d 08:53:37,499 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@359a0bdf 08:53:37,499 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:53:37,499 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:53:37,506 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:37,506 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:37,506 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: became leader with frontend state for [] 08:53:37,506 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Candidate to Leader 08:53:37,506 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6140bee3 08:53:37,506 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Candidate to Leader 08:53:37,507 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 08:53:37,507 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@296a309c 08:53:37,507 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done false 08:53:37,508 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@48014b94 08:53:37,541 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:53:37,555 AM [cluster-test-shard-dispatcher-25] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardReplica (Candidate): Starting new election term 3 08:53:37,557 AM [cluster-test-shard-dispatcher-32] [INFO] Follower - member-3-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:37,557 AM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:37,564 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:37,564 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:37,564 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: became leader with frontend state for [] 08:53:37,564 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Candidate to Leader 08:53:37,564 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6f874cbb 08:53:37,564 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Candidate to Leader 08:53:37,564 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 08:53:37,565 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 08:53:37,565 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:53:37,565 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 08:53:37,565 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@27adcbc1 08:53:37,565 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 08:53:37,566 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardReplica status sync done false 08:53:37,566 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4e5db51 08:53:37,566 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardReplica is ready 08:53:37,566 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} is now ready 08:53:37,566 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-640495729] 08:53:37,566 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:37,566 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2067251677] 08:53:37,567 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2067251677] 08:53:37,567 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#2060258265] 08:53:37,567 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:37,567 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1173054730] 08:53:37,567 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1173054730] 08:53:37,567 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1758622112] 08:53:37,568 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:37,568 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-412253607] 08:53:37,568 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-412253607] 08:53:37,569 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-3}, datastoreType Config 08:53:37,570 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 08:53:37,572 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:37,572 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica, primaryVersion=13] 08:53:37,573 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica for shard member-3-shard-cars-config_testRemoveShardReplica 08:53:37,581 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 08:53:37,581 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 08:53:37,581 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 08:53:37,583 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-640495729] 08:53:37,583 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:37,583 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2067251677] 08:53:37,583 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2067251677] 08:53:37,584 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#2060258265] 08:53:37,584 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:37,584 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1173054730] 08:53:37,584 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#1173054730] 08:53:37,584 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:37,585 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1758622112] 08:53:37,585 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 08:53:37,585 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:37,585 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-412253607] 08:53:37,588 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-412253607] with 27000 ms timeout 08:53:37,590 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 08:53:37,590 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [people] 08:53:37,590 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-config_testRemoveShardReplica: Stopping Shard 08:53:37,591 AM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardReplica: journal closed 08:53:37,591 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica#-412253607] 08:53:37,635 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-1758622112] 08:53:37,635 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:37,635 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [people] 08:53:37,636 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:53:37,636 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardReplica 08:53:37,637 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 08:53:37,637 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:53:37,637 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Stopping Shard 08:53:37,637 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2559] as [Down] 08:53:37,639 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Stopping Shard 08:53:37,639 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal closed 08:53:37,639 AM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal closed 08:53:37,640 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:53:37,640 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:53:37,640 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=0} 08:53:37,640 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 08:53:37,640 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 08:53:37,640 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:53:37,641 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Stopping Shard 08:53:37,641 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Stopping Shard 08:53:37,641 AM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal closed 08:53:37,641 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal closed 08:53:37,641 AM [cluster-test-pekko.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:53:37,641 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:53:37,642 AM [cluster-test-pekko.actor.default-dispatcher-28] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 08:53:37,644 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:37,644 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:37,645 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Node has been marked as DOWN. Shutting down myself 08:53:37,647 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Shutting down... 08:53:37,649 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Successfully shut down 08:53:37,650 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:37,650 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:37,652 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:37,652 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:37,655 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:53:37,677 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:53:37,686 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-8883954872976187116] 08:53:37,686 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 08:53:37,690 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Registered cluster JMX MBean [pekko:type=Cluster] 08:53:37,690 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:53:37,693 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:37,693 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:37,716 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:37,723 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:37,727 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardReplica 08:53:37,746 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1674464023] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:37,746 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:53:37,747 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} started 08:53:37,747 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:53:37,747 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:37,749 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:37,758 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardReplica 08:53:37,762 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica#-1440852003] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:37,763 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:53:37,763 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:53:37,764 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:53:37,764 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardReplica 08:53:37,765 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardReplica 08:53:37,765 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@3addc04d} 08:53:37,765 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:37,765 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardReplica 08:53:37,767 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardReplica 08:53:37,767 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#777682374] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:37,767 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica#-1327130008] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:37,767 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Shard created, persistent : true 08:53:37,768 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#1942996323 created and ready for shard:member-2-shard-people-config_testRemoveShardReplica 08:53:37,769 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:53:37,769 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardReplica 08:53:37,769 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Shard created, persistent : true 08:53:37,769 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardReplica 08:53:37,770 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-7653460 created and ready for shard:member-2-shard-cars-config_testRemoveShardReplica 08:53:37,770 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:37,772 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal open: applyTo=0 08:53:37,773 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:37,773 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica-notifier#1942996323] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:37,775 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica-notifier#-7653460] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:37,775 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:37,776 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@44c22665} 08:53:37,776 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} started 08:53:37,776 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:37,777 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 08:53:37,777 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:53:37,777 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardReplica is initialized 08:53:37,777 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:37,777 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:37,778 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , received role change from null to Follower 08:53:37,778 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:53:37,778 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardReplica from null to Follower 08:53:37,780 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:37,780 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardReplica 08:53:37,780 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardReplica 08:53:37,781 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica#-255257028] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:37,781 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica#2122172514] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:37,781 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Shard created, persistent : true 08:53:37,781 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Shard created, persistent : true 08:53:37,781 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#301252713 created and ready for shard:member-2-shard-people-oper_testRemoveShardReplica 08:53:37,782 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica-notifier#301252713] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:37,782 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:37,782 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#-1353444540 created and ready for shard:member-2-shard-cars-oper_testRemoveShardReplica 08:53:37,782 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica-notifier#-1353444540] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:37,782 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardReplica: Starting recovery with journal batch size 1 08:53:37,783 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal open: applyTo=0 08:53:37,784 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal open: applyTo=0 08:53:37,784 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal open: applyTo=0 08:53:37,803 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:37,805 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:37,808 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:37,809 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:37,810 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:53:37,810 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardReplica is initialized 08:53:37,810 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:37,811 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:37,811 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , received role change from null to Follower 08:53:37,811 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:53:37,811 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardReplica from null to Follower 08:53:37,813 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:37,814 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:53:37,814 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardReplica is initialized 08:53:37,814 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardReplica status sync done true 08:53:37,814 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:37,814 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:37,814 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , received role change from null to Follower 08:53:37,814 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica 08:53:37,814 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardReplica from null to Follower 08:53:37,815 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:37,815 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:37,815 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:37,815 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardReplica status sync done true 08:53:37,815 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:53:37,815 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardReplica is initialized 08:53:37,815 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , received role change from null to Follower 08:53:37,815 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica 08:53:37,815 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardReplica from null to Follower 08:53:37,817 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:53:37,816 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 323, lastApplied : -1, commitIndex : -1 08:53:37,825 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardReplica status sync done true 08:53:37,826 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:53:38,127 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 309, lastApplied : -1, commitIndex : -1 08:53:38,134 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:53:38,145 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:38,325 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:53:38,447 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:38,455 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:38,466 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:38,660 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-2981246241505860661] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:53:38,668 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-2981246241505860661] but current is [-8883954872976187116]. Reason: ActorSystem terminated 08:53:38,698 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#350962407]] intended for system uid [-2981246241505860661], self uid is [-8883954872976187116] 08:53:38,706 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-cars-oper_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 08:53:38,707 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-2-shard-people-config_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-people-config_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 08:53:38,707 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:38,707 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-people-oper_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-people-oper_testRemoveShardReplica, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 08:53:38,708 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Term 3 in "AppendEntries{term=3, leaderId=member-1-shard-cars-config_testRemoveShardReplica, prevLogIndex=0, prevLogTerm=3, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, entries==[]}" message is greater than follower's term 0 - updating term 08:53:38,712 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done false 08:53:38,712 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardReplica status sync done true 08:53:38,712 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@517e4456 08:53:38,712 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): The followers log is empty and the senders prevLogIndex is 0 08:53:38,712 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 08:53:38,713 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@25522f18 08:53:38,713 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done false 08:53:38,713 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardReplica status sync done true 08:53:38,713 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): The followers log is empty and the senders prevLogIndex is 0 08:53:38,713 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@69f24a9e 08:53:38,713 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-config_testRemoveShardReplica (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5} 08:53:38,713 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done false 08:53:38,713 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardReplica status sync done true 08:53:38,713 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6818ffb8 08:53:38,713 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done false 08:53:38,713 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardReplica status sync done true 08:53:38,716 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1216, lastApplied : -1, commitIndex : -1 08:53:38,716 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1132, lastApplied : 0, commitIndex : 0 08:53:38,716 AM [cluster-test-shard-dispatcher-26] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 08:53:38,716 AM [cluster-test-shard-dispatcher-26] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): follower member-2-shard-cars-config_testRemoveShardReplica appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 08:53:38,716 AM [cluster-test-shard-dispatcher-26] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply{term=3, success=false, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 08:53:38,716 AM [cluster-test-shard-dispatcher-26] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader): follower member-2-shard-cars-config_testRemoveShardReplica appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 08:53:38,717 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:38,717 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1224, lastApplied : -1, commitIndex : -1 08:53:38,717 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=true, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 1208, lastApplied : -1, commitIndex : -1 08:53:38,719 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:53:38,768 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:38,774 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:38,786 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:39,86 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:39,87 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:39,88 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 371, lastApplied : -1, commitIndex : -1 08:53:39,94 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:39,95 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 378, lastApplied : -1, commitIndex : -1 08:53:39,95 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:39,104 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:53:39,106 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:39,106 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 388, lastApplied : -1, commitIndex : -1 08:53:39,166 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 444, lastApplied : 0, commitIndex : 0 08:53:39,405 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:39,406 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:53:39,415 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:39,415 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:39,425 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:39,426 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:39,489 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : 0, commitIndex : 0 08:53:39,726 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:39,727 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:39,732 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:39,735 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:39,735 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:39,746 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:39,746 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:39,805 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 312, lastApplied : 0, commitIndex : 0 08:53:40,45 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:40,45 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:40,55 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:40,56 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:40,66 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:40,66 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:40,126 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 08:53:40,345 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#1460627375]] intended for system uid [-2981246241505860661], self uid is [-8883954872976187116] 08:53:40,366 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:40,366 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:40,375 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:40,375 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:40,387 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:40,387 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:40,445 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 312, lastApplied : 0, commitIndex : 0 08:53:40,685 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:40,686 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:40,694 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:40,695 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:40,704 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:53:40,705 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:53:40,752 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:40,766 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 08:53:41,6 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:41,7 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:41,14 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:41,14 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:41,25 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:41,26 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:41,87 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:53:41,326 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:41,327 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:41,334 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:41,335 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:41,344 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:41,344 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:41,406 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 315, lastApplied : 0, commitIndex : 0 08:53:41,645 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:41,645 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:41,655 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:41,655 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:41,665 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:41,666 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:41,725 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 08:53:41,772 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:41,966 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:41,966 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:41,974 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:41,974 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:41,985 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:41,985 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:42,44 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : 0, commitIndex : 0 08:53:42,285 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:42,286 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:42,295 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:42,295 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:42,305 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:42,305 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:42,364 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#1460627375]] intended for system uid [-2981246241505860661], self uid is [-8883954872976187116] 08:53:42,365 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:53:42,603 AM [cluster-test-shard-dispatcher-11] [WARN] Leader - member-1-shard-people-oper_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 08:53:42,604 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:53:42,604 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:42,604 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 08:53:42,604 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 08:53:42,607 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:42,607 AM [cluster-test-shard-dispatcher-11] [INFO] IsolatedLeader - member-1-shard-people-oper_testRemoveShardReplica: IsolatedLeader member-1-shard-people-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 08:53:42,608 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 08:53:42,608 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:42,608 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 08:53:42,608 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from IsolatedLeader to Leader 08:53:42,608 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:53:42,608 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 08:53:42,613 AM [cluster-test-shard-dispatcher-11] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 08:53:42,613 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:53:42,613 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:42,613 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 08:53:42,613 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 08:53:42,615 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:42,615 AM [cluster-test-shard-dispatcher-11] [INFO] IsolatedLeader - member-1-shard-cars-oper_testRemoveShardReplica: IsolatedLeader member-1-shard-cars-oper_testRemoveShardReplica switching from IsolatedLeader to Leader 08:53:42,616 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 08:53:42,616 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:42,616 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:42,616 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from IsolatedLeader to Leader 08:53:42,616 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from IsolatedLeader to Leader 08:53:42,616 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:53:42,623 AM [cluster-test-shard-dispatcher-11] [WARN] Leader - member-1-shard-people-config_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 08:53:42,623 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:53:42,623 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:42,623 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 08:53:42,624 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from Leader to IsolatedLeader 08:53:42,625 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:42,626 AM [cluster-test-shard-dispatcher-11] [INFO] IsolatedLeader - member-1-shard-people-config_testRemoveShardReplica: IsolatedLeader member-1-shard-people-config_testRemoveShardReplica switching from IsolatedLeader to Leader 08:53:42,626 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 08:53:42,626 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:42,626 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 08:53:42,626 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardReplica from IsolatedLeader to Leader 08:53:42,626 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 08:53:42,627 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:42,683 AM [cluster-test-shard-dispatcher-11] [WARN] Leader - member-1-shard-cars-config_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 08:53:42,683 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:53:42,683 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:42,683 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from Leader to IsolatedLeader 08:53:42,683 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from Leader to IsolatedLeader 08:53:42,686 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:53:42,686 AM [cluster-test-shard-dispatcher-11] [INFO] IsolatedLeader - member-1-shard-cars-config_testRemoveShardReplica: IsolatedLeader member-1-shard-cars-config_testRemoveShardReplica switching from IsolatedLeader to Leader 08:53:42,686 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 08:53:42,686 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:42,686 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardReplica , received role change from IsolatedLeader to Leader 08:53:42,686 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardReplica from IsolatedLeader to Leader 08:53:42,686 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardReplica is ready 08:53:42,783 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:42,926 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 08:53:42,926 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 08:53:42,935 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:53:42,935 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:53:42,945 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:53:42,945 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:43,7 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 08:53:43,245 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:43,246 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:43,255 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:43,255 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:43,265 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:43,265 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:43,325 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 08:53:43,424 AM [cluster-test-pekko.actor.default-dispatcher-21] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Down)]. 08:53:43,425 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:43,425 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:43,568 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:43,568 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 08:53:43,575 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:43,575 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:43,584 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:43,584 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:43,645 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:53:43,661 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Down)]. 08:53:43,662 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:43,662 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:43,802 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:43,885 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 08:53:43,886 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:53:43,894 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:43,894 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:43,904 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:43,904 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:43,965 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:53:44,207 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:44,208 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 08:53:44,214 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:44,215 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:44,224 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:44,225 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:44,285 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:53:44,525 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:53:44,525 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:53:44,534 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:44,534 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:53:44,544 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:44,544 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:44,607 AM [cluster-test-shard-dispatcher-26] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : 0, commitIndex : 0 08:53:44,682 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2559] 08:53:44,683 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:44,683 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:44,685 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-2981246241505860661] but current is [-8883954872976187116]. Reason: Cluster member removed, previous status [Down] 08:53:44,822 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2559, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:44,823 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:44,823 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:44,824 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2559] ignored due to non-matching UID, quarantine requested for [-2981246241505860661] but current is [-8883954872976187116]. Reason: Cluster member removed, previous status [Down] 08:53:44,847 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:44,847 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 322, lastApplied : -1, commitIndex : -1 08:53:44,855 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:44,855 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:44,864 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:44,864 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:44,926 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : 0, commitIndex : 0 08:53:45,166 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:45,167 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:45,175 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:45,176 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:45,185 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:45,185 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:45,246 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:53:45,487 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:45,487 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:45,495 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:45,495 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:45,505 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:45,505 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:45,566 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-config_testRemoveShardReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : 0, commitIndex : 0 08:53:45,807 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:45,807 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:45,815 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:53:45,816 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:45,826 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:53:45,826 AM [cluster-test-shard-dispatcher-11] [WARN] AbstractLeader - member-1-shard-people-config_testRemoveShardReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-3-shard-people-config_testRemoveShardReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:53:45,842 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:53:45,847 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:53:45,848 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:45,848 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:53:45,848 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:53:45,848 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:53:45,848 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:53:45,848 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:53:45,848 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:45,848 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:45,848 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-cars-oper_testRemoveShardReplica 08:53:45,848 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:45,848 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardReplica/member-2-shard-people-oper_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardReplica is ready 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-cars-oper_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-cars-oper_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardReplica/member-3-shard-people-oper_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardReplica/member-1-shard-people-oper_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-people-config_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardReplica is ready 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-3-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardReplica/member-3-shard-people-config_testRemoveShardReplica 08:53:45,849 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardReplica: Peer address for peer member-1-shard-people-config_testRemoveShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-people-config_testRemoveShardReplica 08:53:45,849 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} is now ready 08:53:45,849 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1674464023] 08:53:45,850 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:45,850 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#777682374] 08:53:45,850 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#777682374] 08:53:45,851 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-2}, datastoreType Config 08:53:45,851 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-2}] 08:53:45,852 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:45,852 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2067251677, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2d036062] 08:53:45,853 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2067251677 for shard member-2-shard-cars-config_testRemoveShardReplica 08:53:45,853 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardReplica} for shard cars 08:53:45,853 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 08:53:45,853 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 08:53:45,854 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#-640495729] 08:53:45,854 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:45,854 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2067251677] 08:53:45,854 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-1-shard-cars-config_testRemoveShardReplica#2067251677] 08:53:45,854 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1674464023] 08:53:45,854 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:45,854 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#777682374] 08:53:45,855 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-1-shard-cars-config_testRemoveShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:45,856 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#777682374] with 27000 ms timeout 08:53:45,856 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2 : Local Shard replica for shard cars has been removed 08:53:45,856 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [people] 08:53:45,857 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-config_testRemoveShardReplica: Stopping Shard 08:53:45,857 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardReplica: journal closed 08:53:45,858 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica/member-2-shard-cars-config_testRemoveShardReplica#777682374] 08:53:45,904 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardReplica#1674464023] 08:53:45,905 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:45,905 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Local shard cars not found - shards present: [people] 08:53:45,906 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardReplica, generation=1} 08:53:45,906 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardReplica 08:53:45,906 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 08:53:45,907 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testRemoveShardReplica: Stopping Shard 08:53:45,907 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardReplica: journal closed 08:53:45,908 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:53:45,908 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:53:45,909 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardReplica, generation=1} 08:53:45,909 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardReplica 08:53:45,909 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardReplica 08:53:45,909 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:53:45,910 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testRemoveShardReplica: Stopping Shard 08:53:45,910 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardReplica: Stopping Shard 08:53:45,910 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardReplica: journal closed 08:53:45,910 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardReplica: journal closed 08:53:45,911 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:53:45,911 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:53:45,912 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:45,913 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:45,920 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:45,921 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:45,921 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:45,921 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:45,922 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:53:45,932 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:53:45,933 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardReplica 08:53:45,933 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 08:53:45,933 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-config_testRemoveShardReplica: Stopping Shard 08:53:45,934 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardReplica: journal closed 08:53:45,934 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:53:45,934 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:53:45,934 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardReplica, generation=0} 08:53:45,935 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardReplica 08:53:45,935 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardReplica 08:53:45,935 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:53:45,935 AM [cluster-test-shard-dispatcher-34] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardReplica: Stopping Shard 08:53:45,935 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testRemoveShardReplica: Stopping Shard 08:53:45,935 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardReplica: journal closed 08:53:45,935 AM [cluster-test-shard-dispatcher-34] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardReplica: journal closed 08:53:45,936 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:53:45,936 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:53:45,937 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:45,937 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:46,721 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:53:46,934 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-8883954872976187116] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:53:46,953 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [4153471639277276822] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:53:46,954 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:46,955 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:46,957 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:53:46.960] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:46,971 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardReplica, generation=0} 08:53:46,973 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveShardReplica 08:53:46,973 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardReplica 08:53:46,973 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:53:46,973 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@18363fd6 08:53:46,973 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@18363fd6 08:53:46,974 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6bb8d593 08:53:46,974 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6bb8d593 08:53:46,974 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 08:53:46,974 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-1-shard-cars-config_testRemoveShardReplica: Stopping Shard 08:53:46,974 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2eba4680 08:53:46,975 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardReplica: journal closed 08:53:47,422 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: leader transfer aborted 08:53:47,423 AM [cluster-test-shard-dispatcher-26] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveShardReplica: Failed to transfer leadership in 449.1 ms 08:53:47,423 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardReplica: In unpauseLeader 08:53:47,424 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-1-shard-people-config_testRemoveShardReplica: Stopping Shard 08:53:47,424 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardReplica: journal closed 08:53:47,425 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:53:47,425 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:53:47,426 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardReplica, generation=0} 08:53:47,426 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardReplica 08:53:47,427 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardReplica 08:53:47,427 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:53:47,427 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@47cc49b5 08:53:47,427 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1d9fc1ed 08:53:47,427 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6ef023 08:53:47,427 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6ef023 08:53:47,427 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 08:53:47,427 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4a694212 08:53:47,427 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4a694212 08:53:47,427 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: pauseLeader successfully completed - doing transfer 08:53:47,724 AM [cluster-test-shard-dispatcher-25] [WARN] Leader - member-1-shard-people-oper_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 08:53:47,724 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:53:47,724 AM [cluster-test-shard-dispatcher-25] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: leader transfer aborted 08:53:47,724 AM [cluster-test-shard-dispatcher-25] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveShardReplica: Failed to transfer leadership in 297.2 ms 08:53:47,724 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardReplica: In unpauseLeader 08:53:47,724 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:47,725 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 08:53:47,725 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardReplica from Leader to IsolatedLeader 08:53:47,725 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-people-oper_testRemoveShardReplica: Stopping Shard 08:53:47,726 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardReplica: journal closed 08:53:47,732 AM [cluster-test-shard-dispatcher-26] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: leader transfer aborted 08:53:47,733 AM [cluster-test-shard-dispatcher-26] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveShardReplica: Failed to transfer leadership in 305.7 ms 08:53:47,733 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardReplica: In unpauseLeader 08:53:47,733 AM [cluster-test-shard-dispatcher-26] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 08:53:47,734 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:53:47,734 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:47,734 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardReplica , received role change from Leader to IsolatedLeader 08:53:47,734 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardReplica from Leader to IsolatedLeader 08:53:47,734 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardReplica: Stopping Shard 08:53:47,735 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardReplica: journal closed 08:53:47,735 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:53:47,735 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:53:47,737 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:47,737 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:47,741 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:53:47.743] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:47,770 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:53:47,782 AM [cluster-test-pekko.actor.default-dispatcher-8] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [3293859520692599866] 08:53:47,783 AM [cluster-test-pekko.actor.default-dispatcher-8] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:53:47,789 AM [cluster-test-pekko.actor.default-dispatcher-8] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:53:47,789 AM [cluster-test-pekko.actor.default-dispatcher-8] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:53:47,789 AM [cluster-test-pekko.actor.default-dispatcher-8] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:47,789 AM [cluster-test-pekko.actor.default-dispatcher-8] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:47,805 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:53:47,806 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:53:47,806 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:53:47,806 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:47,809 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:47,811 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 08:53:47,816 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-2038581199] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,818 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:53:47,818 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:53:47,829 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:53:47,829 AM [cluster-test-pekko.actor.default-dispatcher-8] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:53:47,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:47,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveAllShardReplicas 08:53:47,830 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:47,830 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:47,830 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:53:47,830 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@31863f90} 08:53:47,830 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 08:53:47,830 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:47,832 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:47,834 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 08:53:47,835 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:47,835 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:47,837 AM [cluster-test-pekko.actor.default-dispatcher-8] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1031356485] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,837 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:53:47,837 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:53:47,838 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:47,838 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:47,838 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas on shard member-1-shard-cars-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1776608934] 08:53:47,839 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveAllShardReplicas 08:53:47,839 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:53:47,839 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:53:47,839 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas on shard member-1-shard-people-config_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-2039159111] 08:53:47,839 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:47,841 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 08:53:47,844 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:47,845 AM [cluster-test-pekko.actor.default-dispatcher-8] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1776608934] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,845 AM [cluster-test-pekko.actor.default-dispatcher-8] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-2039159111] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,845 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#478370781 created and ready for shard:member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:47,845 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#125023985 created and ready for shard:member-1-shard-people-config_testRemoveAllShardReplicas 08:53:47,845 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas-notifier#478370781] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,845 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas-notifier#125023985] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,847 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:53:47,847 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:47,847 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-1-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:47,848 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:47,848 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:47,848 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:47,848 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:47,849 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@364a6bf4} 08:53:47,849 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 08:53:47,852 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:47,852 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:47,852 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:47,854 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:47,854 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:47,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:47,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-1-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#-81009638] 08:53:47,854 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:47,854 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:47,854 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:47,856 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:47,856 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:47,856 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-1-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#139457975] 08:53:47,856 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:47,857 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:53:47,857 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveAllShardReplicas is initialized 08:53:47,856 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:47,858 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#-1934909211 created and ready for shard:member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:47,858 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:47,858 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:47,858 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas#139457975] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,858 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas-notifier#-1934909211] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,859 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:47,861 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:47,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:47,865 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:47,865 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:47,865 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 08:53:47,865 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas#-81009638] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,865 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#1213271126 created and ready for shard:member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:47,866 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:47,866 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas-notifier#1213271126] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,866 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:53:47,866 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:53:47,866 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveAllShardReplicas is initialized 08:53:47,866 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from null to Follower 08:53:47,866 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:47,867 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:47,868 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:47,869 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:47,869 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:47,869 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 08:53:47,869 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:53:47,869 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from null to Follower 08:53:47,869 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:47,869 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:47,869 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:53:47,869 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveAllShardReplicas is initialized 08:53:47,869 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:47,869 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 08:53:47,870 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 08:53:47,870 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from null to Follower 08:53:47,870 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:47,870 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:53:47,870 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveAllShardReplicas is initialized 08:53:47,870 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:47,870 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:47,870 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 08:53:47,870 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 08:53:47,870 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 08:53:47,877 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:53:47,890 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-777602462206116519] 08:53:47,890 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 08:53:47,893 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:47,893 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:53:47,900 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:47,901 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:47,917 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:47,920 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:47,922 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 08:53:47,929 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1358309333] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,930 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:53:47,931 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:53:47,940 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:53:47,940 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveAllShardReplicas 08:53:47,941 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveAllShardReplicas 08:53:47,942 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:53:47,943 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@e434ef8} 08:53:47,943 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 08:53:47,943 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:47,943 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveAllShardReplicas 08:53:47,943 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:47,946 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:47,950 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 08:53:47,956 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveAllShardReplicas 08:53:47,958 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1933987674] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,958 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:47,960 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#1965915257] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,962 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:53:47,962 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:53:47,963 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:53:47,963 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveAllShardReplicas 08:53:47,964 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveAllShardReplicas 08:53:47,965 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:47,965 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#1108468102 created and ready for shard:member-2-shard-people-config_testRemoveAllShardReplicas 08:53:47,965 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-2-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:47,967 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActor - member-2-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:47,967 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#-1137182125] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,967 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-902293977] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,967 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas-notifier#1108468102] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,972 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 08:53:47,972 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:47,973 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:47,974 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@5a813bba} 08:53:47,974 AM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:47,978 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas-notifier#1965915257 created and ready for shard:member-2-shard-cars-config_testRemoveAllShardReplicas 08:53:47,978 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:47,982 AM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:47,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:47,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:47,983 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 08:53:47,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:53:47,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveAllShardReplicas is initialized 08:53:47,983 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:53:47,983 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:47,983 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveAllShardReplicas from null to Follower 08:53:47,984 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:47,984 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:47,984 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 08:53:47,985 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:53:47,985 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveAllShardReplicas is initialized 08:53:47,988 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:53:47,989 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 08:53:47,989 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveAllShardReplicas from null to Follower 08:53:47,989 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:47,989 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveAllShardReplicas 08:53:47,994 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveAllShardReplicas 08:53:47,994 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas#-1010960487] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,994 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas#-2001049196] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,994 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:47,995 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#-754190627 created and ready for shard:member-2-shard-people-oper_testRemoveAllShardReplicas 08:53:47,995 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas-notifier#-754190627] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:47,995 AM [cluster-test-shard-dispatcher-20] [INFO] RaftActor - member-2-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:47,997 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:47,998 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:53:47,999 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:48,2 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:53:48,2 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:48,3 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:48,3 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,3 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,3 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:48,3 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:53:48,3 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:48,3 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:48,3 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:53:48,3 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:53:48,3 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:53:48,3 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:48,3 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-2-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:53:48,4 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:48,5 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:53:48,5 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveAllShardReplicas is initialized 08:53:48,5 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:48,5 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:48,5 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 08:53:48,5 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:48,5 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 08:53:48,5 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveAllShardReplicas from null to Follower 08:53:48,7 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#-887967721 created and ready for shard:member-2-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,8 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas-notifier#-887967721] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:48,8 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:48,9 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:48,10 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:53:48,12 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:48,17 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:48,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:53:48,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveAllShardReplicas is initialized 08:53:48,17 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:48,17 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:48,17 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,18 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-8582685664547342825] 08:53:48,19 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 08:53:48,19 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 08:53:48,19 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 08:53:48,20 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 08:53:48,21 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:48,22 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:53:48,23 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:48,23 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:48,36 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:48,38 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:48,40 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveAllShardReplicas 08:53:48,50 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:53:48,50 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:53:48,52 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1014035021] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:48,53 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:53:48,54 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveAllShardReplicas 08:53:48,54 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveAllShardReplicas 08:53:48,57 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:53:48,60 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@5b601d03} 08:53:48,60 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} started 08:53:48,61 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:48,63 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:48,65 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:48,65 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveAllShardReplicas 08:53:48,65 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveAllShardReplicas 08:53:48,66 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:48,66 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#615571354] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:48,67 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#1850110084 created and ready for shard:member-3-shard-cars-config_testRemoveAllShardReplicas 08:53:48,67 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:48,67 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas-notifier#1850110084] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:48,68 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:48,71 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-1022546075] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:48,72 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:48,73 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:48,73 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:53:48,75 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveAllShardReplicas 08:53:48,75 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:48,75 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#600172941 created and ready for shard:member-3-shard-people-config_testRemoveAllShardReplicas 08:53:48,76 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas-notifier#600172941] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:48,76 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:53:48,76 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-people-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:48,77 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:48,78 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:53:48,78 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveAllShardReplicas is initialized 08:53:48,78 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:48,78 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:48,78 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:53:48,78 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:53:48,78 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:53:48,79 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:48,80 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:48,80 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:48,80 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:48,80 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , received role change from null to Follower 08:53:48,80 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:53:48,81 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas#1397937207] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:48,81 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:53:48,81 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:53:48,82 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:48,83 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:53:48,83 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveAllShardReplicas is initialized 08:53:48,83 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:48,83 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:48,83 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , received role change from null to Follower 08:53:48,83 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:53:48,83 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:53:48,83 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveAllShardReplicas from null to Follower 08:53:48,90 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveAllShardReplicas from null to Follower 08:53:48,91 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:53:48,91 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveAllShardReplicas 08:53:48,92 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:48,92 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,92 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:48,93 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveAllShardReplicas: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 08:53:48,94 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@331b78a6} 08:53:48,94 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} started 08:53:48,94 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:53:48,94 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:48,94 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,95 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,95 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,95 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas on shard member-3-shard-cars-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-900683783] 08:53:48,95 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveAllShardReplicas 08:53:48,95 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:48,95 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-900683783] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:48,95 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:53:48,95 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveAllShardReplicas on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas on shard member-3-shard-people-oper_testRemoveAllShardReplicas whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-1260005201] 08:53:48,96 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-1260005201] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:48,96 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:48,96 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:48,96 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#1351725916 created and ready for shard:member-3-shard-people-oper_testRemoveAllShardReplicas 08:53:48,96 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-people-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:48,97 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas-notifier#1351725916] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:48,98 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-111540120 created and ready for shard:member-3-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,98 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas-notifier#-111540120] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:48,98 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:48,98 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:48,99 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:48,101 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:48,102 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:53:48,102 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:48,102 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:48,102 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , received role change from null to Follower 08:53:48,102 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:48,102 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveAllShardReplicas is initialized 08:53:48,102 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 08:53:48,102 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveAllShardReplicas from null to Follower 08:53:48,106 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:48,107 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:48,110 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:48,110 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:53:48,110 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:48,110 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:48,110 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveAllShardReplicas is initialized 08:53:48,111 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , received role change from null to Follower 08:53:48,111 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-1-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,111 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas 08:53:48,111 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveAllShardReplicas from null to Follower 08:53:48,240 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 08:53:48,240 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:48,240 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:48,241 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:48,241 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 08:53:48,241 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Follower to Candidate 08:53:48,269 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 08:53:48,270 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:48,270 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:48,270 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:48,270 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 08:53:48,270 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Follower to Candidate 08:53:48,271 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 08:53:48,271 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:48,271 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:48,271 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:48,271 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 08:53:48,271 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Follower to Candidate 08:53:48,290 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 1 08:53:48,290 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:48,290 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:48,290 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:48,290 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Follower to Candidate 08:53:48,290 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Follower to Candidate 08:53:48,621 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 08:53:48,621 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 08:53:48,664 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 2 08:53:48,681 AM [cluster-test-shard-dispatcher-14] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 2 08:53:48,808 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:53:48,809 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:53:48,810 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:48,810 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:53:48,810 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:48,810 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:53:48,810 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:53:48,810 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:48,810 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:53:48,810 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:48,811 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-1-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:48,811 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,811 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,812 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:53:48,812 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:48,820 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:48,820 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:48,820 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-3-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:53:48,820 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:48,820 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,821 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:53:48,821 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,821 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:53:48,821 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:53:48,821 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:53:48,821 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:53:48,821 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:53:48,933 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:48,934 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:48,934 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:48,934 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:48,934 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-cars-oper_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-cars-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas 08:53:48,935 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-config_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-config_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas 08:53:48,934 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveAllShardReplicas with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas to pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas 08:53:48,936 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveAllShardReplicas: Peer address for peer member-2-shard-people-oper_testRemoveAllShardReplicas set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveAllShardReplicas/member-2-shard-people-oper_testRemoveAllShardReplicas 08:53:48,971 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 08:53:48,973 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:48,975 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:48,983 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:48,983 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 08:53:48,983 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@38be919 08:53:48,984 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: became leader with frontend state for [] 08:53:48,984 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 08:53:48,984 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveAllShardReplicas from Candidate to Leader 08:53:48,985 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1b1b016d 08:53:48,985 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done false 08:53:48,986 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1dea81ee 08:53:48,986 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveAllShardReplicas status sync done false 08:53:49,12 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 08:53:49,14 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:49,15 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-people-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:49,23 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:49,23 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 08:53:49,23 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 08:53:49,23 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 08:53:49,23 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4388e310 08:53:49,23 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveAllShardReplicas from Candidate to Leader 08:53:49,24 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveAllShardReplicas status sync done false 08:53:49,24 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done false 08:53:49,24 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4894b55f 08:53:49,24 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@31b5f538 08:53:49,30 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate): Starting new election term 3 08:53:49,31 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:49,31 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-people-config_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:49,34 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:53:49,41 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:49,41 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 08:53:49,41 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: became leader with frontend state for [] 08:53:49,41 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 08:53:49,42 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@68a7505 08:53:49,42 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveAllShardReplicas from Candidate to Leader 08:53:49,42 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 08:53:49,42 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} is now ready 08:53:49,42 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-2038581199] 08:53:49,42 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:49,42 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1776608934] 08:53:49,42 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1776608934] 08:53:49,42 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveAllShardReplicas status sync done false 08:53:49,43 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done false 08:53:49,43 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1a76895c 08:53:49,43 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 08:53:49,43 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1b1bd58d 08:53:49,43 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 08:53:49,43 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1358309333] 08:53:49,43 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:49,43 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1933987674] 08:53:49,44 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1933987674] 08:53:49,44 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1014035021] 08:53:49,44 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:49,44 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#615571354] 08:53:49,44 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#615571354] 08:53:49,56 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@4b74de3c] 08:53:49,57 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1 doCreateShard: shardId: member-1-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 08:53:49,58 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:49,59 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActor - member-1-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:49,59 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas-notifier#1043746902 created and ready for shard:member-1-shard-pets-config_testRemoveAllShardReplicas 08:53:49,60 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:49,62 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-1-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:49,68 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-1-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:49,69 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [pets] 08:53:49,69 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-pets-config_testRemoveAllShardReplicas is initialized 08:53:49,69 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:49,69 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:49,69 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 08:53:49,69 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:53:49,69 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from null to Follower 08:53:49,69 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@7ed7fe13] 08:53:49,70 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2 doCreateShard: shardId: member-2-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, member-3-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 08:53:49,72 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:49,72 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas-notifier#742412560 created and ready for shard:member-2-shard-pets-config_testRemoveAllShardReplicas 08:53:49,73 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:49,73 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: onCreateShard: CreateShard [moduleShardConfig=ModuleShardConfiguration [namespace=pets-ns, moduleName=pets-module, shardName=pets, shardMemberNames=[MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], shardStrategyName=null], shardPropsCreator=org.opendaylight.controller.cluster.datastore.Shard$Builder@5023b0f] 08:53:49,73 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 doCreateShard: shardId: member-3-shard-pets-config_testRemoveAllShardReplicas, memberNames: [MemberName{name=member-1}, MemberName{name=member-2}, MemberName{name=member-3}], peerAddresses: {member-2-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas, member-1-shard-pets-config_testRemoveAllShardReplicas=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas}, isActiveMember: true 08:53:49,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-2038581199] 08:53:49,74 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Shard created, persistent : true 08:53:49,74 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 08:53:49,74 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#-2103656876] 08:53:49,75 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#-2103656876] 08:53:49,75 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas-notifier#-712956054 created and ready for shard:member-3-shard-pets-config_testRemoveAllShardReplicas 08:53:49,75 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-pets-config_testRemoveAllShardReplicas: Starting recovery with journal batch size 1 08:53:49,75 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1358309333] 08:53:49,75 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 08:53:49,75 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:49,76 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-pets-config_testRemoveAllShardReplicas: journal open: applyTo=0 08:53:49,78 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:49,78 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-pets-config_testRemoveAllShardReplicas: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:49,76 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 08:53:49,84 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:49,84 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [pets] 08:53:49,84 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is initialized 08:53:49,84 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:49,84 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:49,85 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 08:53:49,85 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:53:49,85 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-pets-config_testRemoveAllShardReplicas from null to Follower 08:53:49,86 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-pets-config_testRemoveAllShardReplicas: Local TermInfo store seeded with TermInfo{term=0} 08:53:49,87 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [pets] 08:53:49,87 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testRemoveAllShardReplicas is initialized 08:53:49,87 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-pets-config_testRemoveAllShardReplicas is initialized - notifying 1 OnShardInitialized callbacks 08:53:49,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:49,87 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:49,87 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , received role change from null to Follower 08:53:49,87 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1927843701] 08:53:49,87 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-pets-config_testRemoveAllShardReplicas , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas 08:53:49,87 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1927843701] 08:53:49,87 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-pets-config_testRemoveAllShardReplicas from null to Follower 08:53:49,88 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1014035021] 08:53:49,88 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 08:53:49,88 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#1527149429] 08:53:49,88 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#1527149429] 08:53:49,90 AM [cluster-test-shard-dispatcher-25] [INFO] Candidate - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate): Starting new election term 3 08:53:49,91 AM [main] [INFO] ClusterAdminRpcService - Removing replicas for all shards 08:53:49,92 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people, pets] for data store config_testRemoveAllShardReplicas 08:53:49,92 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testRemoveAllShardReplicas 08:53:49,92 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:49,92 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 08:53:49,92 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-3}] 08:53:49,93 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 08:53:49,93 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:49,93 AM [cluster-test-shard-dispatcher-22] [INFO] Follower - member-2-shard-cars-oper_testRemoveAllShardReplicas (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:49,94 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=pets, memberName=MemberName{name=member-3}] 08:53:49,94 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-oper_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 08:53:49,94 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:49,94 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas, primaryVersion=13] 08:53:49,94 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: onRemoveShardReplica: RemoveShardReplica [shardName=people, memberName=MemberName{name=member-3}] 08:53:49,94 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 08:53:49,94 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 08:53:49,94 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas, primaryVersion=13] 08:53:49,94 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for people: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas, primaryVersion=13] 08:53:49,95 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=pets, waitUntilReady=true] 08:53:49,95 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-pets-config_testRemoveAllShardReplicas, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 08:53:49,95 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas for shard member-3-shard-cars-config_testRemoveAllShardReplicas 08:53:49,96 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas for shard member-3-shard-people-config_testRemoveAllShardReplicas 08:53:49,96 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-people-oper_testRemoveAllShardReplicas for shard member-3-shard-people-oper_testRemoveAllShardReplicas 08:53:49,100 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:53:49,100 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveAllShardReplicas} for shard cars 08:53:49,100 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard cars 08:53:49,100 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 08:53:49,100 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-config_testRemoveAllShardReplicas} for shard people 08:53:49,100 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard people 08:53:49,100 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:53:49,100 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:53:49,101 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 1 08:53:49,101 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@bcd3d9c 08:53:49,100 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: became leader with frontend state for [] 08:53:49,101 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveAllShardReplicas , received role change from Candidate to Leader 08:53:49,101 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveAllShardReplicas from Candidate to Leader 08:53:49,101 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 08:53:49,101 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 2 08:53:49,101 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-people-oper_testRemoveAllShardReplicas} for shard people 08:53:49,102 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard people 08:53:49,102 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveAllShardReplicas status sync done false 08:53:49,102 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:53:49,102 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 3 08:53:49,102 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@52ac1f38 08:53:49,102 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 08:53:49,102 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas, primaryVersion=13] 08:53:49,102 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 08:53:49,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveAllShardReplicas/member-1-shard-cars-oper_testRemoveAllShardReplicas for shard member-3-shard-cars-oper_testRemoveAllShardReplicas 08:53:49,104 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done false 08:53:49,104 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6c4e6f3a 08:53:49,104 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveAllShardReplicas is ready 08:53:49,106 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveAllShardReplicas status sync done true 08:53:49,106 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testRemoveAllShardReplicas} for shard cars 08:53:49,106 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully removed the replica shard cars 08:53:49,106 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:53:49,106 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 4 08:53:49,106 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:49,107 AM [cluster-test-shard-dispatcher-25] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:49,107 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-people-oper_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:49,108 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-1022546075] with 27000 ms timeout 08:53:49,108 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-1260005201] with 27000 ms timeout 08:53:49,108 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard people has been removed 08:53:49,108 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets, cars] 08:53:49,108 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard people has been removed 08:53:49,108 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 08:53:49,109 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 08:53:49,109 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 08:53:49,109 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveAllShardReplicas status sync done true 08:53:49,109 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveAllShardReplicas: journal closed 08:53:49,109 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveAllShardReplicas status sync done true 08:53:49,110 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveAllShardReplicas: journal closed 08:53:49,110 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-people-config_testRemoveAllShardReplicas#-1022546075] 08:53:49,110 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-people-oper_testRemoveAllShardReplicas#-1260005201] 08:53:49,112 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:49,112 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#615571354] with 27000 ms timeout 08:53:49,112 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard cars has been removed 08:53:49,112 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [pets] 08:53:49,112 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveAllShardReplicas status sync done true 08:53:49,112 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 08:53:49,113 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveAllShardReplicas: journal closed 08:53:49,113 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-cars-config_testRemoveAllShardReplicas#615571354] 08:53:49,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-900683783] with 27000 ms timeout 08:53:49,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3 : Local Shard replica for shard cars has been removed 08:53:49,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [] 08:53:49,114 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 08:53:49,114 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveAllShardReplicas: journal closed 08:53:49,115 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveAllShardReplicas/member-3-shard-cars-oper_testRemoveAllShardReplicas#-900683783] 08:53:49,420 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate): Starting new election term 1 08:53:49,420 AM [cluster-test-shard-dispatcher-14] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveAllShardReplicas (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveAllShardReplicas, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 309, lastApplied : 0, commitIndex : 0 08:53:49,421 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-pets-config_testRemoveAllShardReplicas (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:49,421 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: false, hasLeader : false 08:53:49,421 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:49,421 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Follower to Candidate 08:53:49,421 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Follower to Candidate 08:53:49,422 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-3-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:49,422 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-2-shard-pets-config_testRemoveAllShardReplicas (Follower): Term 1 in "RequestVote{term=1, candidateId=member-1-shard-pets-config_testRemoveAllShardReplicas, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:53:49,430 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-pets-config_testRemoveAllShardReplicas (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:49,430 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: onLeadershipChange, isLeader: true, hasLeader : true 08:53:49,430 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: became leader with frontend state for [] 08:53:49,430 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@25741ab6 08:53:49,431 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-pets-config_testRemoveAllShardReplicas , received role change from Candidate to Leader 08:53:49,431 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-pets-config_testRemoveAllShardReplicas from Candidate to Leader 08:53:49,431 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 08:53:49,431 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done false 08:53:49,431 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-pets-config_testRemoveAllShardReplicas status sync done false 08:53:49,432 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@17de1f5d 08:53:49,432 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-3-shard-pets-config_testRemoveAllShardReplicas is ready - notifying 1 OnShardInitialized callbacks 08:53:49,432 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for pets: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas, primaryVersion=13] 08:53:49,432 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 08:53:49,432 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas for shard member-3-shard-pets-config_testRemoveAllShardReplicas 08:53:49,432 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5f4349f0 08:53:49,433 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveAllShardReplicas is ready 08:53:49,435 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-pets-config_testRemoveAllShardReplicas} for shard pets 08:53:49,435 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully removed the replica shard pets 08:53:49,436 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=pets}, type Config 08:53:49,436 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 5, actual 5 08:53:49,438 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-2038581199] 08:53:49,438 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:49,438 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1776608934] 08:53:49,438 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-cars-config_testRemoveAllShardReplicas#-1776608934] 08:53:49,439 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-2038581199] 08:53:49,439 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:53:49,439 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-2039159111] 08:53:49,439 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-people-config_testRemoveAllShardReplicas#-2039159111] 08:53:49,439 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-2038581199] 08:53:49,439 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : pets 08:53:49,439 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#-2103656876] 08:53:49,439 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-1-shard-pets-config_testRemoveAllShardReplicas#-2103656876] 08:53:49,439 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1358309333] 08:53:49,440 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:49,440 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1933987674] 08:53:49,440 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-cars-config_testRemoveAllShardReplicas#1933987674] 08:53:49,440 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1358309333] 08:53:49,440 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:53:49,440 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-902293977] 08:53:49,440 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-people-config_testRemoveAllShardReplicas#-902293977] 08:53:49,440 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActorVotingConfigSupport - member-1-shard-pets-config_testRemoveAllShardReplicas: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-pets-config_testRemoveAllShardReplicas, isVoting=true], ServerInfo[peerId=member-1-shard-pets-config_testRemoveAllShardReplicas, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:49,440 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1358309333] 08:53:49,440 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : pets 08:53:49,441 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1927843701] 08:53:49,441 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-2-shard-pets-config_testRemoveAllShardReplicas#-1927843701] 08:53:49,441 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#1527149429] with 27000 ms timeout 08:53:49,441 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3 : Local Shard replica for shard pets has been removed 08:53:49,441 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [] 08:53:49,442 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 08:53:49,442 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-pets-config_testRemoveAllShardReplicas: journal closed 08:53:49,443 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1014035021] 08:53:49,443 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-pets-config_testRemoveAllShardReplicas status sync done true 08:53:49,443 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas/member-3-shard-pets-config_testRemoveAllShardReplicas#1527149429] 08:53:49,443 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:49,443 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard cars not found - shards present: [] 08:53:49,444 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1014035021] 08:53:49,444 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:53:49,444 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard people not found - shards present: [] 08:53:49,444 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveAllShardReplicas#-1014035021] 08:53:49,444 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : pets 08:53:49,444 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: Local shard pets not found - shards present: [] 08:53:49,444 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:53:49,445 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 0 shards 08:53:49,445 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:53:49,445 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:53:49,445 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 08:53:49,446 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 0 shards 08:53:49,446 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:53:49,446 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:53:49,447 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:49,447 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:49,452 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:49,452 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:49,452 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:49,453 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:49,455 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:53:49,465 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:53:49,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-pets-config_testRemoveAllShardReplicas 08:53:49,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveAllShardReplicas 08:53:49,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveAllShardReplicas 08:53:49,466 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 3 shards 08:53:49,466 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-2-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 08:53:49,466 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 08:53:49,466 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 08:53:49,466 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-2-shard-pets-config_testRemoveAllShardReplicas: journal closed 08:53:49,467 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveAllShardReplicas: journal closed 08:53:49,468 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveAllShardReplicas: journal closed 08:53:49,468 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:53:49,468 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:53:49,469 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 08:53:49,469 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveAllShardReplicas 08:53:49,469 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveAllShardReplicas 08:53:49,469 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:53:49,470 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 08:53:49,470 AM [cluster-test-shard-dispatcher-20] [INFO] Shard - member-2-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 08:53:49,470 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveAllShardReplicas: journal closed 08:53:49,471 AM [cluster-test-shard-dispatcher-20] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveAllShardReplicas: journal closed 08:53:49,471 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:53:49,471 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:53:49,472 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:49,472 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:49,827 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:53:50,467 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-8582685664547342825] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:53:50,488 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-777602462206116519] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:53:50,495 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:50,496 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:50,499 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:53:50.503] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:50,514 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveAllShardReplicas, generation=0} 08:53:50,515 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-pets-config_testRemoveAllShardReplicas 08:53:50,515 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testRemoveAllShardReplicas 08:53:50,516 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveAllShardReplicas 08:53:50,516 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 3 shards 08:53:50,516 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@c3bd308 08:53:50,516 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-pets-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@c3bd308 08:53:50,516 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 08:53:50,516 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3819abde 08:53:50,516 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2856a8d4 08:53:50,516 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-people-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2856a8d4 08:53:50,517 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 08:53:50,517 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3dbdb7c1 08:53:50,517 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3dbdb7c1 08:53:50,517 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 08:53:50,517 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@51b8f43f 08:53:50,517 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@58e98cce 08:53:50,897 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: leader transfer aborted 08:53:50,897 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveAllShardReplicas: Failed to transfer leadership in 380.6 ms 08:53:50,897 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: In unpauseLeader 08:53:50,898 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-cars-config_testRemoveAllShardReplicas: Stopping Shard 08:53:50,899 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveAllShardReplicas: journal closed 08:53:50,957 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: leader transfer aborted 08:53:50,957 AM [cluster-test-shard-dispatcher-14] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testRemoveAllShardReplicas: Failed to transfer leadership in 440.5 ms 08:53:50,957 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: In unpauseLeader 08:53:50,958 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-config_testRemoveAllShardReplicas: Stopping Shard 08:53:50,958 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveAllShardReplicas: journal closed 08:53:51,27 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: leader transfer aborted 08:53:51,28 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-pets-config_testRemoveAllShardReplicas: Failed to transfer leadership in 512.2 ms 08:53:51,28 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: In unpauseLeader 08:53:51,28 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-1-shard-pets-config_testRemoveAllShardReplicas: Stopping Shard 08:53:51,29 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-1-shard-pets-config_testRemoveAllShardReplicas: journal closed 08:53:51,30 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:53:51,30 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:53:51,30 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveAllShardReplicas, generation=0} 08:53:51,31 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveAllShardReplicas 08:53:51,31 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveAllShardReplicas 08:53:51,31 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:53:51,32 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2ddb8b56 08:53:51,32 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c7ab7f3 08:53:51,32 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c7ab7f3 08:53:51,32 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 08:53:51,32 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@62eb904e 08:53:51,32 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-people-oper_testRemoveAllShardReplicas: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@62eb904e 08:53:51,32 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: pauseLeader successfully completed - doing transfer 08:53:51,32 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7d882293 08:53:51,337 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: leader transfer aborted 08:53:51,337 AM [cluster-test-shard-dispatcher-24] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 306.0 ms 08:53:51,338 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: In unpauseLeader 08:53:51,338 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveAllShardReplicas: Stopping Shard 08:53:51,339 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveAllShardReplicas: journal closed 08:53:51,577 AM [cluster-test-shard-dispatcher-14] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: leader transfer aborted 08:53:51,577 AM [cluster-test-shard-dispatcher-14] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testRemoveAllShardReplicas: Failed to transfer leadership in 545.7 ms 08:53:51,577 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: In unpauseLeader 08:53:51,578 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-1-shard-people-oper_testRemoveAllShardReplicas: Stopping Shard 08:53:51,579 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveAllShardReplicas: journal closed 08:53:51,580 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:53:51,580 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:53:51,581 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:51,581 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:51,584 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:53:51.587] [cluster-test-pekko.actor.internal-dispatcher-5] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:51,615 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:53:51,623 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [8076804373905148039] 08:53:51,623 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:53:51,631 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:53:51,631 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:53:51,631 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:51,631 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:51,639 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:51,640 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:51,642 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:53:51,642 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:53:51,642 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:53:51,644 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplicaFailures 08:53:51,648 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:53:51,648 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#25957641] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,648 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:53:51,651 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:53:51,651 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplicaFailures 08:53:51,652 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:51,652 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 08:53:51,654 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 08:53:51,655 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@561435d6} 08:53:51,655 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} started 08:53:51,656 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:51,656 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:51,656 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplicaFailures 08:53:51,657 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Shard created, persistent : true 08:53:51,657 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#1236487028] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,657 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#1599960818] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,658 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures-notifier#1599960818 created and ready for shard:member-1-shard-cars-config_testAddShardReplicaFailures 08:53:51,658 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-config_testAddShardReplicaFailures: Starting recovery with journal batch size 1 08:53:51,659 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:51,659 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplicaFailures: journal open: applyTo=0 08:53:51,662 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplicaFailures 08:53:51,662 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 08:53:51,662 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures 08:53:51,662 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures on shard member-1-shard-cars-config_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures/member-1-shard-cars-config_testAddShardReplicaFailures#1236487028] 08:53:51,663 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:51,663 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures#-1862661324] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,664 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:53:51,664 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:53:51,665 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:53:51,665 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplicaFailures 08:53:51,666 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:51,666 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 08:53:51,667 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplicaFailures: Local TermInfo store seeded with TermInfo{term=0} 08:53:51,667 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:53:51,667 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplicaFailures is initialized 08:53:51,667 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 08:53:51,668 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:51,668 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplicaFailures: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 08:53:51,668 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from null to Follower 08:53:51,668 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@58d3c164} 08:53:51,669 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} started 08:53:51,670 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:51,670 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplicaFailures 08:53:51,670 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate): Starting new election term 1 08:53:51,670 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:51,670 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 08:53:51,670 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:51,670 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Follower to Candidate 08:53:51,670 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:51,670 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 08:53:51,670 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: became leader with frontend state for [] 08:53:51,670 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , received role change from Candidate to Leader 08:53:51,670 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplicaFailures , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures 08:53:51,670 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 08:53:51,670 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplicaFailures with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures to pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures 08:53:51,670 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplicaFailures on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures on shard member-1-shard-cars-oper_testAddShardReplicaFailures whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-1289513341] 08:53:51,671 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Shard created, persistent : true 08:53:51,671 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures#-1289513341] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,671 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#222144856 created and ready for shard:member-1-shard-cars-oper_testAddShardReplicaFailures 08:53:51,672 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures/member-1-shard-cars-oper_testAddShardReplicaFailures-notifier#222144856] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,678 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5cec5ea6 08:53:51,672 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=people} 08:53:51,678 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplicaFailures from Candidate to Leader 08:53:51,678 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplicaFailures is ready 08:53:51,678 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onAddShardReplica: AddShardReplica[ShardName=people] 08:53:51,678 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: No module configuration exists for shard people 08:53:51,678 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-oper_testAddShardReplicaFailures: Starting recovery with journal batch size 1 08:53:51,678 AM [cluster-test-pekko.actor.default-dispatcher-13] [ERROR] ClusterAdminRpcService - Failed to add replica for shard ShardName{value=people} java.lang.IllegalArgumentException: No module configuration exists for shard people at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onAddShardReplica(ShardManager.java:1164) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:248) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:33) at org.apache.pekko.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:29) at scala.PartialFunction.applyOrElse(PartialFunction.scala:225) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:224) at org.apache.pekko.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:29) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:280) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:281) at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547) at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545) at org.apache.pekko.persistence.AbstractPersistentActor.org$apache$pekko$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:305) at org.apache.pekko.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:931) at org.apache.pekko.persistence.Eventsourced.aroundReceive(Eventsourced.scala:256) at org.apache.pekko.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:255) at org.apache.pekko.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:305) at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590) at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557) at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280) at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241) at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 08:53:51,679 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplicaFailures, generation=0} 08:53:51,680 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplicaFailures 08:53:51,680 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@19eecb5f 08:53:51,680 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplicaFailures: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@19eecb5f 08:53:51,680 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 08:53:51,680 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplicaFailures: Stopping Shard 08:53:51,680 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplicaFailures: journal open: applyTo=0 08:53:51,681 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplicaFailures: journal closed 08:53:51,681 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:53:51,681 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:53:51,681 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplicaFailures, generation=0} 08:53:51,681 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#25957641] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#25957641] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,681 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplicaFailures 08:53:51,681 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 08:53:51,682 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplicaFailures: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:51,682 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#25957641] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplicaFailures#25957641] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,682 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007fccc082b6c8] to Actor[pekko://cluster-test/user/$a#1351829378] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#1351829378] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,684 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplicaFailures: Local TermInfo store seeded with TermInfo{term=0} 08:53:51,685 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:53:51,685 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplicaFailures is initialized 08:53:51,685 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 08:53:51,685 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:51,685 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from null to Follower 08:53:51,687 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate): Starting new election term 1 08:53:51,687 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplicaFailures (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:51,688 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: false, hasLeader : false 08:53:51,688 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:51,688 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Follower to Candidate 08:53:51,688 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplicaFailures (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:51,688 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplicaFailures: onLeadershipChange, isLeader: true, hasLeader : true 08:53:51,688 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: became leader with frontend state for [] 08:53:51,688 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , received role change from Candidate to Leader 08:53:51,688 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplicaFailures , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplicaFailures 08:53:51,688 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5af2c647 08:53:51,688 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplicaFailures from Candidate to Leader 08:53:51,688 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplicaFailures is ready 08:53:51,688 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplicaFailures: Stopping Shard 08:53:51,689 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplicaFailures: journal closed 08:53:51,689 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:53:51,689 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:53:51,690 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007fccc082b6c8] to Actor[pekko://cluster-test/user/$b#-1314686442] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/$b#-1314686442] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,690 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:51,691 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:51,692 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:53:51,717 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:53:51,729 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-3813289931876990719] 08:53:51,729 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:53:51,737 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:53:51,737 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:53:51,737 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:51,737 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:51,750 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:51,752 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:53:51,752 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:53:51,753 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:53:51,755 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:51,758 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 08:53:51,763 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1905444515] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,764 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:53:51,764 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:53:51,770 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:53:51,770 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testAddShardReplica 08:53:51,771 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:51,771 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:53:51,772 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 08:53:51,773 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@4dd0a56d} 08:53:51,773 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} started 08:53:51,774 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:51,774 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testAddShardReplica 08:53:51,775 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:53:51,775 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:53:51,775 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica on shard member-1-shard-cars-config_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501] 08:53:51,775 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,775 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Shard created, persistent : true 08:53:51,776 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#-738385193 created and ready for shard:member-1-shard-cars-config_testAddShardReplica 08:53:51,776 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-notifier#-738385193] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,776 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 08:53:51,778 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplica: journal open: applyTo=0 08:53:51,781 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:51,784 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:51,785 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:53:51,785 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testAddShardReplica is initialized 08:53:51,785 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:51,785 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:51,785 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from null to Follower 08:53:51,788 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-config_testAddShardReplica (Candidate): Starting new election term 1 08:53:51,788 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:51,788 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:51,788 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:51,788 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Follower to Candidate 08:53:51,788 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:51,789 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:51,789 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: became leader with frontend state for [] 08:53:51,789 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , received role change from Candidate to Leader 08:53:51,789 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 08:53:51,789 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7e1e39b 08:53:51,789 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testAddShardReplica from Candidate to Leader 08:53:51,789 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 08:53:51,824 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1905444515] 08:53:51,824 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:51,824 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501] 08:53:51,825 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:51,827 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:51,831 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 08:53:51,833 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1797590499] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,835 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:53:51,835 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:53:51,836 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 08:53:51,836 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:53:51,836 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testAddShardReplica 08:53:51,836 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} started 08:53:51,837 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:51,837 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 08:53:51,837 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@77b52a63} 08:53:51,837 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:51,837 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testAddShardReplica 08:53:51,837 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 08:53:51,837 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 08:53:51,837 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testAddShardReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica on shard member-1-shard-cars-oper_testAddShardReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423] 08:53:51,838 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,838 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 08:53:51,838 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#1318047755 created and ready for shard:member-1-shard-cars-oper_testAddShardReplica 08:53:51,839 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-notifier#1318047755] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,839 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 08:53:51,840 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 08:53:51,841 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:51,843 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:51,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:51,843 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:53:51,843 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:51,843 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testAddShardReplica is initialized 08:53:51,843 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from null to Follower 08:53:51,845 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testAddShardReplica (Candidate): Starting new election term 1 08:53:51,845 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:53:51,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:51,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:51,845 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Follower to Candidate 08:53:51,845 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:53:51,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:53:51,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: became leader with frontend state for [] 08:53:51,845 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Candidate to Leader 08:53:51,845 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 08:53:51,845 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@720b66f5 08:53:51,846 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Candidate to Leader 08:53:51,846 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:51,887 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1797590499] 08:53:51,887 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:53:51,887 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423] 08:53:51,896 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:53:51,912 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [5415657185059666992] 08:53:51,912 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 08:53:51,917 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:51,917 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:53:51,921 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:51,921 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:51,928 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:51,929 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:51,931 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 08:53:51,940 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#2017644314] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,940 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:53:51,940 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:53:51,952 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:53:51,955 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 08:53:51,956 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@7fe1cd1b} 08:53:51,956 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} started 08:53:51,957 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:51,958 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:51,960 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:51,961 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 08:53:51,962 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#828830133] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:51,962 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:53:51,962 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:53:51,963 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:53:51,969 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 08:53:51,969 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@71262c97} 08:53:51,969 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:53:51,970 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} started 08:53:51,970 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:51,972 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:53:51,972 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:51,972 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:51,972 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:51,973 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 08:53:52,753 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:53:52,754 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:52,754 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 08:53:52,754 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:52,754 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 08:53:52,754 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:53:52,754 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 08:53:52,754 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 08:53:52,755 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:52,772 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:53:52,772 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 08:53:52,772 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:52,773 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica] 08:53:52,774 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:53:52,775 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501, primaryVersion=13] 08:53:52,777 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501 for shard member-2-shard-cars-config_testAddShardReplica 08:53:52,778 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Shard created, persistent : true 08:53:52,779 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-2-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 08:53:52,780 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddShardReplica: journal open: applyTo=0 08:53:52,780 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#602987121 created and ready for shard:member-2-shard-cars-config_testAddShardReplica 08:53:52,780 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:52,781 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-notifier#602987121] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:52,781 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-config_testAddShardReplica 08:53:52,782 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:53:52,782 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:52,782 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:52,787 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: snapshot is durable as of 2026-04-10T08:53:52.782244588Z 08:53:52,787 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-2-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:52,788 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:53:52,788 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testAddShardReplica is initialized 08:53:52,788 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:52,788 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:52,788 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , received role change from null to Follower 08:53:52,788 AM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-2-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:53:52,790 AM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-2-shard-cars-config_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddShardReplica 08:53:52,791 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-2-shard-cars-config_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:52,791 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 08:53:52,791 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 08:53:52,791 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7519a66b 08:53:52,791 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testAddShardReplica from null to Follower 08:53:52,791 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 08:53:52,793 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-2-shard-cars-config_testAddShardReplica: snapshot is durable as of 2026-04-10T08:53:52.791523780Z 08:53:52,793 AM [cluster-test-shard-dispatcher-14] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddShardReplica: Applying snapshot 08:53:52,793 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:52,794 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#-194479752] with name member-2-shard-cars-config_testAddShardReplica-DTCL-publisher 08:53:52,794 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#-194479752] 08:53:52,794 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: state snapshot applied in 506.4 μs 08:53:52,794 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 08:53:52,794 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 81.56 μs 08:53:52,794 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica-DTCL-publisher#-194479752] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:52,794 AM [cluster-test-shard-dispatcher-14] [INFO] ShardSnapshotCohort - member-2-shard-cars-config_testAddShardReplica: Done applying snapshot 08:53:52,795 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:53:52,798 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501 08:53:52,798 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Leader shard successfully added the replica shard cars 08:53:52,798 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-config_testAddShardReplica 08:53:52,798 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: persisting the shard list [cars] 08:53:52,800 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:52,802 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@301b26ac 08:53:52,802 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 08:53:52,802 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:53:52,802 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#2017644314] 08:53:52,802 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 08:53:52,802 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:52,802 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:52,803 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:52,803 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 08:53:52,803 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:52,803 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#828830133] 08:53:52,803 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:53:52,803 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Local shard cars not found - shards present: [] 08:53:52,803 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:53:52,803 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: onAddShardReplica: AddShardReplica[ShardName=cars] 08:53:52,804 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:52,804 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica] 08:53:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:53:52,806 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423, primaryVersion=13] 08:53:52,807 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423 for shard member-2-shard-cars-oper_testAddShardReplica 08:53:52,808 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 08:53:52,808 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1347148732] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:52,808 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#124642071 created and ready for shard:member-2-shard-cars-oper_testAddShardReplica 08:53:52,808 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 08:53:52,809 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-notifier#124642071] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:52,810 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 08:53:52,811 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=-1, lastAppliedTerm=-1, lastIndex=-1, lastTerm=-1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-oper_testAddShardReplica 08:53:52,811 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:52,811 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Persising snapshot at EntryInfo[index=-1, term=-1]/EntryInfo[index=-1, term=-1] 08:53:52,811 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:52,813 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-2-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:52,814 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:52,814 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:52,814 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , received role change from null to Follower 08:53:52,814 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=-1, lastIncludedTerm=-1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1]}" message is greater than follower's term 0 - updating term 08:53:52,815 AM [cluster-test-shard-dispatcher-28] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2026-04-10T08:53:52.811754116Z 08:53:52,816 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:53:52,816 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testAddShardReplica is initialized 08:53:52,817 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-2-shard-cars-oper_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddShardReplica 08:53:52,818 AM [cluster-test-shard-dispatcher-12] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:52,818 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 08:53:52,818 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 08:53:52,818 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@115bcdb0 08:53:52,819 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testAddShardReplica from null to Follower 08:53:52,819 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:52,821 AM [cluster-test-shard-dispatcher-14] [INFO] SnapshotManager - member-2-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2026-04-10T08:53:52.818409146Z 08:53:52,821 AM [cluster-test-shard-dispatcher-14] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddShardReplica: Applying snapshot 08:53:52,821 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:52,821 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#977091660] with name member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher 08:53:52,821 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#977091660] 08:53:52,821 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: state snapshot applied in 643.0 μs 08:53:52,822 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher#977091660] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:52,822 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 08:53:52,822 AM [cluster-test-shard-dispatcher-14] [INFO] ShardSnapshotCohort - member-2-shard-cars-oper_testAddShardReplica: Done applying snapshot 08:53:52,822 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 331.4 μs 08:53:52,823 AM [cluster-test-shard-dispatcher-28] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-2-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to -1, nextIndex set to 0 08:53:52,825 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423 08:53:52,825 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Leader shard successfully added the replica shard cars 08:53:52,826 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-oper_testAddShardReplica 08:53:52,826 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: persisting the shard list [cars] 08:53:52,829 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:52,830 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1692cbdd 08:53:52,830 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:53:52,830 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:52,830 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 08:53:52,830 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#828830133] 08:53:52,830 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 08:53:52,830 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:53:52,830 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:52,830 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1347148732] 08:53:52,830 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1347148732] 08:53:52,838 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:53:52,849 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-4634437355387546402] 08:53:52,850 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 08:53:52,853 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:53:52,853 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:53:52,854 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:52,854 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:52,868 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:52,872 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:52,875 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 08:53:52,882 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#128771145] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:52,882 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:53:52,883 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:53:52,893 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:53:52,894 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:53:52,894 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 08:53:52,895 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:53:52,896 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:52,897 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:53:52,897 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:52,897 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:53:52,897 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@6f2070d0} 08:53:52,897 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} started 08:53:52,898 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:52,898 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:52,899 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:52,901 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testAddShardReplica 08:53:52,902 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1308886949] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:52,902 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:53:52,903 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:53:52,904 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:53:52,904 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:52,904 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:52,904 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:52,904 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:52,906 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 08:53:52,907 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@61e78344} 08:53:52,907 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} started 08:53:52,907 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:52,938 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:52,939 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:52,939 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 08:53:52,939 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 08:53:52,939 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 08:53:52,939 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 08:53:52,939 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 08:53:52,939 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:53,415 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done false 08:53:53,444 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done false 08:53:53,734 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testAddShardReplica status sync done true 08:53:53,765 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testAddShardReplica status sync done true 08:53:53,773 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:53,773 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 08:53:53,773 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 08:53:53,773 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:53,773 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:53,773 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:53:53,773 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:53:53,773 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 08:53:53,773 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:53:54,793 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:53:54,798 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:54,798 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:53:54,798 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:54,798 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:54,885 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:53:54,890 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:54,890 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:53:54,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 08:53:54,890 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testAddShardReplica is ready 08:53:54,890 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:54,890 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 08:53:54,890 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica 08:53:54,890 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:54,912 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:53:54,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 08:53:54,913 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:54,913 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica] 08:53:54,915 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:53:54,915 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501, primaryVersion=13] 08:53:54,917 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501 for shard member-3-shard-cars-config_testAddShardReplica 08:53:54,918 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Shard created, persistent : true 08:53:54,918 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#291317752] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:54,919 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#1700448494 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 08:53:54,919 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-3-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 08:53:54,919 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#1700448494] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:54,921 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal open: applyTo=0 08:53:54,923 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-config_testAddShardReplica 08:53:54,923 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-3-shard-cars-config_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:54,923 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 08:53:54,923 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:54,926 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-3-shard-cars-config_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:54,927 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:53:54,927 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 08:53:54,927 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:54,927 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:54,927 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 08:53:54,928 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-config_testAddShardReplica: snapshot is durable as of 2026-04-10T08:53:54.923828856Z 08:53:54,928 AM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-3-shard-cars-config_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-config_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]}}" message is greater than follower's term 0 - updating term 08:53:54,930 AM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-3-shard-cars-config_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-config_testAddShardReplica 08:53:54,931 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 08:53:54,931 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@65d81ba0 08:53:54,931 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 08:53:54,931 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:53:54,931 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 08:53:54,931 AM [cluster-test-shard-dispatcher-21] [INFO] SnapshotManager - member-3-shard-cars-config_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:54,934 AM [cluster-test-shard-dispatcher-21] [INFO] SnapshotManager - member-3-shard-cars-config_testAddShardReplica: snapshot is durable as of 2026-04-10T08:53:54.931712501Z 08:53:54,934 AM [cluster-test-shard-dispatcher-21] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testAddShardReplica: Applying snapshot 08:53:54,934 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:54,934 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#992274264] 08:53:54,935 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#992274264] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:54,935 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#992274264] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 08:53:54,935 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 08:53:54,935 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 73.29 μs 08:53:54,935 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 1.175 ms 08:53:54,935 AM [cluster-test-shard-dispatcher-21] [INFO] ShardSnapshotCohort - member-3-shard-cars-config_testAddShardReplica: Done applying snapshot 08:53:54,938 AM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-config_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-config_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 08:53:54,941 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-config_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501 08:53:54,941 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Leader shard successfully added the replica shard cars 08:53:54,941 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-config_testAddShardReplica 08:53:54,941 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: persisting the shard list [cars] 08:53:54,942 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-config_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:54,946 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@20da897f 08:53:54,946 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:53:54,946 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:53:54,946 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#128771145] 08:53:54,946 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:54,946 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#291317752] 08:53:54,946 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#291317752] 08:53:54,948 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1308886949] 08:53:54,948 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:53:54,948 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Local shard cars not found - shards present: [] 08:53:54,948 AM [main] [INFO] ClusterAdminRpcService - Adding replica for shard ShardName{value=cars} 08:53:54,948 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: onAddShardReplica: AddShardReplica[ShardName=cars] 08:53:54,948 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:54,948 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: findPrimary for cars forwarding to remote ShardManager pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica, visitedAddresses: [pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica] 08:53:54,949 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 08:53:54,949 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:54,949 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 08:53:54,950 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.RemoteFindPrimary [shardName=cars, waitUntilReady=true] 08:53:54,950 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423, primaryVersion=13] 08:53:54,953 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Shard created, persistent : true 08:53:54,954 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-3-shard-cars-oper_testAddShardReplica: Starting recovery with journal batch size 1 08:53:54,956 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testAddShardReplica: journal open: applyTo=0 08:53:54,956 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#20258358 created and ready for shard:member-3-shard-cars-oper_testAddShardReplica 08:53:54,956 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-219203604] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:54,957 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-notifier#20258358] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:54,957 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending AddServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423 for shard member-3-shard-cars-oper_testAddShardReplica 08:53:54,959 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-3-shard-cars-oper_testAddShardReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:53:54,959 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=1, lastIndex=0, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-3-shard-cars-oper_testAddShardReplica 08:53:54,959 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Persising snapshot at EntryInfo[index=0, term=1]/EntryInfo[index=0, term=1] 08:53:54,959 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: Removed in-memory snapshotted entries, adjusted snapshotIndex: -1 and term: -1 08:53:54,962 AM [cluster-test-shard-dispatcher-11] [INFO] SnapshotManager - member-1-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2026-04-10T08:53:54.959759500Z 08:53:54,963 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-3-shard-cars-oper_testAddShardReplica: Local TermInfo store seeded with TermInfo{term=0} 08:53:54,963 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:54,964 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:54,964 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , received role change from null to Follower 08:53:54,964 AM [cluster-test-shard-dispatcher-19] [INFO] Follower - member-3-shard-cars-oper_testAddShardReplica (Follower): Term 1 in "InstallSnapshot{term=1, leaderId=member-1-shard-cars-oper_testAddShardReplica, lastIncludedIndex=0, lastIncludedTerm=1, datasize=229, chunk=1/1, lastChunkHashCode=OptionalInt[-1], votingConfig=VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]}}" message is greater than follower's term 0 - updating term 08:53:54,964 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:53:54,964 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testAddShardReplica is initialized 08:53:54,967 AM [cluster-test-shard-dispatcher-19] [INFO] Follower - member-3-shard-cars-oper_testAddShardReplica (Follower): Snapshot received from leader: member-1-shard-cars-oper_testAddShardReplica 08:53:54,967 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testAddShardReplica 08:53:54,967 AM [cluster-test-shard-dispatcher-19] [INFO] SnapshotManager - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot on follower: PlainSnapshotSource{io=MemoryStreamSource{size=229}} 08:53:54,967 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5f2300ed 08:53:54,967 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testAddShardReplica from null to Follower 08:53:54,967 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:54,968 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 08:53:54,970 AM [cluster-test-shard-dispatcher-19] [INFO] SnapshotManager - member-3-shard-cars-oper_testAddShardReplica: snapshot is durable as of 2026-04-10T08:53:54.968061843Z 08:53:54,971 AM [cluster-test-shard-dispatcher-19] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testAddShardReplica: Applying snapshot 08:53:54,971 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:54,971 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#1372732187] 08:53:54,971 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#1372732187] with name member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher 08:53:54,972 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 08:53:54,972 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher#1372732187] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:54,972 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 104.2 μs 08:53:54,972 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: state snapshot applied in 1.078 ms 08:53:54,972 AM [cluster-test-shard-dispatcher-19] [INFO] ShardSnapshotCohort - member-3-shard-cars-oper_testAddShardReplica: Done applying snapshot 08:53:54,973 AM [cluster-test-shard-dispatcher-11] [INFO] AbstractLeader - member-1-shard-cars-oper_testAddShardReplica (Leader): Snapshot successfully installed on follower member-3-shard-cars-oper_testAddShardReplica (last chunk 1) - matchIndex set to 0, nextIndex set to 1 08:53:54,976 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Received AddServerReply{status=OK, leaderHint=member-1-shard-cars-oper_testAddShardReplica} for shard cars from leader pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423 08:53:54,976 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Leader shard successfully added the replica shard cars 08:53:54,977 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testAddShardReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testAddShardReplica, isVoting=true], ServerInfo[peerId=member-1-shard-cars-oper_testAddShardReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:53:54,977 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-3-shard-cars-oper_testAddShardReplica 08:53:54,977 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: persisting the shard list [cars] 08:53:54,984 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@51e4f579 08:53:54,984 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClusterAdminRpcService - Successfully added replica for shard ShardName{value=cars} 08:53:54,984 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testAddShardReplica is ready 08:53:54,984 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1308886949] 08:53:54,984 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:53:54,984 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-219203604] 08:53:54,984 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-219203604] 08:53:54,985 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#2017644314] 08:53:54,986 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:54,986 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:54,986 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:54,986 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#828830133] 08:53:54,986 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:53:54,986 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1347148732] 08:53:54,986 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1347148732] 08:53:54,991 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : true 08:53:54,991 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-3-shard-cars-oper_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:54,991 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 08:53:55,76 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#2017644314] 08:53:55,77 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:55,78 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 08:53:55,79 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 08:53:55,97 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 08:53:55,110 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state LeaderFrontendState{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=126922, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 08:53:55,117 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 08:53:55,122 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 08:53:55,147 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 27.65 ms 08:53:55,150 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,176 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: readying transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,176 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 ready 08:53:55,180 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: transitioned member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 to ready 08:53:55,180 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 08:53:55,181 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Validating transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,181 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 validated 08:53:55,181 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 08:53:55,181 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 08:53:55,182 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Preparing transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,185 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 candidate ready 08:53:55,186 AM [cluster-test-shard-dispatcher-11] [DEBUG] UserCohorts - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 08:53:55,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] UserCohorts - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 08:53:55,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] UserCohorts - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 08:53:55,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 prepared 08:53:55,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 08:53:55,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 08:53:55,192 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Starting commit for transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,197 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 08:53:55,200 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 08:53:55,203 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 08:53:55,204 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Resuming commit of transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,206 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@37155a3b}, purged=MutableUnsignedLongSet{size=0}} 08:53:55,206 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,206 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,206 AM [cluster-test-shard-dispatcher-11] [DEBUG] UserCohorts - member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 08:53:55,206 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitCohort - Transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 08:53:55,213 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#1981751238] with name member-1-shard-cars-config_testAddShardReplica-DTCL-publisher 08:53:55,213 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#1981751238] 08:53:55,214 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 08:53:55,214 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica-DTCL-publisher#1981751238] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:55,214 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-config_testAddShardReplica-DTCL-publisher: 195.3 μs 08:53:55,218 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#128771145] 08:53:55,218 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:55,219 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 08:53:55,219 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 08:53:55,228 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,228 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,229 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@560e70f6}, purged=MutableUnsignedLongSet{size=0}} 08:53:55,229 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,229 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,229 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 08:53:55,230 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 112.6 μs 08:53:55,230 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 08:53:55,230 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-config_testAddShardReplica: Applying foreign transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,230 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@28402b7}, purged=MutableUnsignedLongSet{size=0}} 08:53:55,230 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,230 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,230 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 08:53:55,230 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-config_testAddShardReplica-DTCL-publisher: 67.13 μs 08:53:55,235 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} is not yet known 08:53:55,235 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state LeaderFrontendState{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=10331, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 08:53:55,239 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 08:53:55,239 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 08:53:55,240 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 742.4 μs 08:53:55,241 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: replication of member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007fccc09e0680@77604788 08:53:55,242 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: finished purging transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,243 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,243 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,244 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,244 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,249 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,258 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1308886949] 08:53:55,258 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:55,258 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 08:53:55,258 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 08:53:55,260 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 08:53:55,260 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state LeaderFrontendState{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=17220, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 08:53:55,261 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 08:53:55,261 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 08:53:55,262 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-3-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 799.0 μs 08:53:55,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: readying transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 ready 08:53:55,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: transitioned member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 to ready 08:53:55,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct canCommit 08:53:55,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Validating transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 validated 08:53:55,265 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 08:53:55,265 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct preCommit 08:53:55,265 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Preparing transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,265 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 candidate ready 08:53:55,265 AM [cluster-test-shard-dispatcher-11] [DEBUG] UserCohorts - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - candidate rootPath: / 08:53:55,265 AM [cluster-test-shard-dispatcher-11] [DEBUG] UserCohorts - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: canCommit - messages: [] 08:53:55,265 AM [cluster-test-shard-dispatcher-11] [DEBUG] UserCohorts - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 08:53:55,265 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 prepared 08:53:55,265 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 08:53:55,265 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 initiating direct commit 08:53:55,265 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Starting commit for transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,266 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 233 08:53:55,266 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 submitted to persistence 08:53:55,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Resuming commit of transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@74cd2d62}, purged=MutableUnsignedLongSet{size=0}} 08:53:55,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] UserCohorts - member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1: commit - successfulFromPrevious: [] 08:53:55,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitCohort - Transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 08:53:55,270 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done false 08:53:55,270 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-oper_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#1479564977] with name member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher 08:53:55,270 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#1479564977] 08:53:55,271 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 08:53:55,271 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher#1479564977] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:55,271 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-1-shard-cars-oper_testAddShardReplica-DTCL-publisher: 168.4 μs 08:53:55,271 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#828830133] 08:53:55,271 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:55,271 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 08:53:55,271 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica, primaryVersion=13] 08:53:55,272 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-3-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,272 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-3-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1a9280bd}, purged=MutableUnsignedLongSet{size=0}} 08:53:55,273 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,273 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,273 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 08:53:55,273 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-oper_testAddShardReplica-DTCL-publisher: 127.7 μs 08:53:55,273 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testAddShardReplica status sync done true 08:53:55,273 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,274 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} is not yet known 08:53:55,274 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: created state LeaderFrontendState{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, nanosAgo=11460, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 08:53:55,275 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 08:53:55,275 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 08:53:55,275 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-oper_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 339.5 μs 08:53:55,276 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007fccc09e0680@71a50c2f 08:53:55,276 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,276 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,276 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,276 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: allocating new open snapshot member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,277 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,277 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,279 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#1905444515] 08:53:55,279 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:53:55,279 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501] 08:53:55,279 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501] 08:53:55,280 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#-1797590499] 08:53:55,280 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:53:55,280 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423] 08:53:55,280 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423] 08:53:55,280 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#2017644314] 08:53:55,280 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:53:55,280 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:55,280 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:55,331 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:55,381 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:55,432 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:55,483 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:55,534 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:55,585 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:55,635 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:55,654 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,654 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,686 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica#-582311007] 08:53:55,686 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#828830133] 08:53:55,686 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:53:55,686 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1347148732] 08:53:55,686 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1347148732] 08:53:55,705 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-2-shard-cars-oper_testAddShardReplica: Applying foreign transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,705 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4a583da5}, purged=MutableUnsignedLongSet{size=0}} 08:53:55,705 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,705 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Committed transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,705 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-oper_testAddShardReplica: publishChanges: rootPath: / 08:53:55,706 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-oper_testAddShardReplica: Elapsed time for generation of change events for member-2-shard-cars-oper_testAddShardReplica-DTCL-publisher: 112.7 μs 08:53:55,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,706 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-3-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,737 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-2-shard-cars-oper_testAddShardReplica#1347148732] 08:53:55,737 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#128771145] 08:53:55,737 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:55,737 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#291317752] 08:53:55,737 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#291317752] 08:53:55,738 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica#1308886949] 08:53:55,738 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:53:55,738 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-219203604] 08:53:55,738 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica#-219203604] 08:53:55,738 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0} 08:53:55,739 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node [pekko://cluster-test@127.0.0.1:2557] as [Down] 08:53:55,739 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 08:53:55,739 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 08:53:55,741 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Stopping Shard 08:53:55,741 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal closed 08:53:55,742 AM [cluster-test-pekko.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:53:55,742 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:53:55,743 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testAddShardReplica, generation=0} 08:53:55,743 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007fccc082b6c8] to Actor[pekko://cluster-test/user/$a#-374836427] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#-374836427] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:55,744 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testAddShardReplica 08:53:55,744 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 1 shards 08:53:55,744 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-oper_testAddShardReplica: Stopping Shard 08:53:55,745 AM [cluster-test-pekko.actor.default-dispatcher-27] [WARN] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Received gossip where this member has been downed, from [pekko://cluster-test@127.0.0.1:2558] 08:53:55,745 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#128771145] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#128771145] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:55,745 AM [cluster-test-pekko.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Node has been marked as DOWN. Shutting down myself 08:53:55,745 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Shutting down... 08:53:55,746 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Successfully shut down 08:53:55,746 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:55,746 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:53:55,746 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberRemoved: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:53:55,746 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] CoordinatedShutdown - Running CoordinatedShutdown with reason [ClusterDowningReason] 08:53:55,748 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testAddShardReplica: journal closed 08:53:55,748 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:53:55,748 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:53:55,748 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:53:55,748 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:53:55,755 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:55,755 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:55,756 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:55,756 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:53:55,758 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:53:55,772 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:53:55,789 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-8990183271398413318] 08:53:55,789 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 08:53:55,792 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Registered cluster JMX MBean [pekko:type=Cluster] 08:53:55,792 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:53:55,792 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:53:55,792 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:53:55,806 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:55,808 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:53:55,815 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:53:55,817 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testAddShardReplica 08:53:55,819 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-0-txn-0-0} 08:53:55,820 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-0} 08:53:55,822 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: aborting transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:55,829 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1924817535] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:55,829 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:53:55,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: applying snapshot ShardManagerSnapshot [ShardList = [cars] ] 08:53:55,830 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: adding shard cars 08:53:55,841 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:53:55,841 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testAddShardReplica 08:53:55,845 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@6ae7a9b5} 08:53:55,845 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} started 08:53:55,846 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:53:55,846 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testAddShardReplica 08:53:55,846 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1924817535] 08:53:55,846 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:53:55,846 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 300000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=false, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 08:53:55,847 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1313894032] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:55,847 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Shard created, persistent : true 08:53:55,848 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#330623158 created and ready for shard:member-3-shard-cars-config_testAddShardReplica 08:53:55,848 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-notifier#330623158] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:55,848 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActor - member-3-shard-cars-config_testAddShardReplica: Starting recovery with journal batch size 1 08:53:55,856 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal open: applyTo=3 08:53:55,858 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: Applying recovered snapshot 08:53:55,863 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 08:53:55,865 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-config_testAddShardReplica: Created publisher actor Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-216784938] with name member-3-shard-cars-config_testAddShardReplica-DTCL-publisher 08:53:55,865 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-3-shard-cars-config_testAddShardReplica: state snapshot applied in 2.029 ms 08:53:55,865 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - Recovery snapshot applied for member-3-shard-cars-config_testAddShardReplica in 6.891 ms: snapshotIndex=0, snapshotTerm=1, journal-size=0 08:53:55,866 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-3-shard-cars-config_testAddShardReplica: Recovery completed in 7.046 ms - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = 0, snapshot term = 1, journal size = 0 08:53:55,868 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-216784938] 08:53:55,869 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-config_testAddShardReplica: publishChanges: rootPath: / 08:53:55,869 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica-DTCL-publisher#-216784938] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:55,869 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-config_testAddShardReplica: Elapsed time for generation of change events for member-3-shard-cars-config_testAddShardReplica-DTCL-publisher: 144.4 μs 08:53:55,870 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 08:53:55,876 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-3-shard-cars-config_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7caa60}, purged=MutableUnsignedLongSet{size=0}} 08:53:55,876 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,876 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Committed transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,876 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardRecoveryCoordinator - member-3-shard-cars-config_testAddShardReplica: starting log recovery batch with max size 1 08:53:55,877 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:55,877 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-config_testAddShardReplica: Purged transaction member-2-datastore-config_testAddShardReplica-fe-0-txn-0-1 08:53:55,877 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:53:55,877 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:53:55,877 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized 08:53:55,877 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-3-shard-cars-config_testAddShardReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:53:55,877 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 08:53:55,877 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , received role change from null to Follower 08:53:55,878 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1313894032] 08:53:55,878 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica#-1313894032] 08:53:55,878 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#-1924817535] 08:53:55,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:53:55,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Scheduling 18000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-config_testAddShardReplica, leaderAvailable=false, actorInitialized=true, followerSyncStatus=false, role=null, leaderId=null, activeMember=true] 08:53:55,879 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testAddShardReplica , registered listener pekko://cluster-test/user/shardmanager-config_testAddShardReplica 08:53:55,879 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testAddShardReplica from null to Follower 08:53:55,879 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 08:53:56,26 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda/0x00007fccc09e8be8@56fcae9b 08:53:56,26 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7dd39ada}, purged=MutableUnsignedLongSet{size=0}} 08:53:56,27 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:56,27 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Aborted transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:56,31 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-2-shard-cars-oper_testAddShardReplica: Added client FrontendClientMetadataBuilder{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7a7a4445}, purged=MutableUnsignedLongSet{size=0}} 08:53:56,32 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:56,32 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Aborted transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:56,33 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: purging transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:56,35 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: replication of member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda/0x00007fccc09e0680@6827bb84 08:53:56,36 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-oper_testAddShardReplica: finished purging transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:56,36 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:56,36 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-oper_testAddShardReplica: Purged transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:56,344 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0}, history=0, cookie=1} 08:53:56,344 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-oper_testAddShardReplica: Purged transaction member-2-datastore-oper_testAddShardReplica-fe-0-txn-0-1 08:53:56,762 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-4634437355387546402] but current is [-8990183271398413318]. Reason: ActorSystem terminated 08:53:56,763 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-4634437355387546402] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:53:56,847 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done false 08:53:56,847 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@251d40a2 08:53:56,847 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is initialized - notifying 1 OnShardInitialized callbacks 08:53:56,847 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:56,847 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testAddShardReplica status sync done true 08:53:56,848 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:56,848 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:56,849 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:56,849 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:56,984 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:57,254 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:53:57,311 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:57,624 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423] to Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:53:57,831 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:58,851 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:59,823 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2558/system/clusterReceptionist/replicator#1770764608]] intended for system uid [-4634437355387546402], self uid is [-8990183271398413318] 08:53:59,870 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:53:59,893 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Down)]. 08:53:59,894 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:59,895 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:59,898 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:53:59,898 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:00,4 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[pekko://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-1213635211]] intended for system uid [-4634437355387546402], self uid is [-8990183271398413318] 08:54:00,75 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Down)]. 08:54:00,891 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - New incarnation of existing member [Member(pekko://cluster-test@127.0.0.1:2557, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 08:54:00,913 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is removing unreachable node [pekko://cluster-test@127.0.0.1:2557] 08:54:00,914 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:00,914 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:00,914 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-4634437355387546402] but current is [-8990183271398413318]. Reason: Cluster member removed, previous status [Down] 08:54:01,98 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:01,98 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberRemoved: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:01,98 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Association - pekkoQuarantine Quarantine of [pekko://cluster-test@127.0.0.1:2557] ignored due to non-matching UID, quarantine requested for [-4634437355387546402] but current is [-8990183271398413318]. Reason: Cluster member removed, previous status [Down] 08:54:01,911 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:54:01,914 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:54:01,914 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:01,915 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:54:01,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:54:01,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testAddShardReplica is ready - notifying 1 OnShardInitialized callbacks 08:54:01,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Found primary for cars: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 08:54:01,915 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:54:01,915 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:01,915 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 08:54:01,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:54:01,915 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testAddShardReplica is ready 08:54:01,915 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-1-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:54:01,915 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-config_testAddShardReplica: Peer address for peer member-2-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testAddShardReplica/member-2-shard-cars-config_testAddShardReplica 08:54:01,915 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica, primaryVersion=13] 08:54:01,918 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: retiring state LeaderFrontendState{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=0}, nanosAgo=6668878755, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 08:54:01,920 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: created state LeaderFrontendState{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, nanosAgo=21750, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 08:54:01,922 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolved shard 1 to ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 08:54:01,922 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 08:54:01,923 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] ClientActorBehavior - member-3-frontend-datastore-config_testAddShardReplica: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 719.1 μs 08:54:01,925 AM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-config_testAddShardReplica: allocating new open snapshot member-3-datastore-config_testAddShardReplica-fe-1-txn-0-1 08:54:01,927 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testAddShardReplica, generation=1} 08:54:01,928 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testAddShardReplica 08:54:01,928 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 1 shards 08:54:01,929 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-3-shard-cars-config_testAddShardReplica: Stopping Shard 08:54:01,929 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testAddShardReplica: journal closed 08:54:01,930 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:54:01,930 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:54:01,931 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:01,931 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:01,937 AM [cluster-test-pekko.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:01,938 AM [cluster-test-pekko.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:01,939 AM [cluster-test-pekko.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:01,939 AM [cluster-test-pekko.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:01,944 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:54:01,950 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testAddShardReplica, generation=0} 08:54:01,950 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testAddShardReplica 08:54:01,951 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 1 shards 08:54:01,951 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testAddShardReplica: Stopping Shard 08:54:01,952 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testAddShardReplica: journal closed 08:54:01,952 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:54:01,952 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:54:01,953 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testAddShardReplica, generation=0} 08:54:01,953 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#2017644314] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#2017644314] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:01,953 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testAddShardReplica 08:54:01,953 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#2017644314] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica#2017644314] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:01,954 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 1 shards 08:54:01,954 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-oper_testAddShardReplica: Stopping Shard 08:54:01,954 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testAddShardReplica: journal closed 08:54:01,954 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:54:01,954 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:54:01,955 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:01,956 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:02,953 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-8990183271398413318] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:02,953 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:54:02,954 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:02,954 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 08:54:02,954 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica to pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica 08:54:02,954 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testAddShardReplica is ready 08:54:02,955 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:02,955 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-1-shard-cars-oper_testAddShardReplica: Peer address for peer member-3-shard-cars-oper_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testAddShardReplica/member-3-shard-cars-oper_testAddShardReplica 08:54:02,955 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:54:02,955 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testAddShardReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica to pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica 08:54:02,955 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testAddShardReplica is ready 08:54:02,955 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-1-shard-cars-config_testAddShardReplica: Peer address for peer member-3-shard-cars-config_testAddShardReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testAddShardReplica/member-3-shard-cars-config_testAddShardReplica 08:54:02,972 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [5415657185059666992] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:02,977 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:02,977 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:02,979 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:54:02.981] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:02,994 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testAddShardReplica, generation=0} 08:54:02,994 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testAddShardReplica 08:54:02,994 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 08:54:02,995 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@74c35ae5 08:54:02,995 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a0f29f3 08:54:02,995 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-config_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a0f29f3 08:54:02,995 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: pauseLeader successfully completed - doing transfer 08:54:03,973 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:54:08,53 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2559, Up)]. 08:54:08,54 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:08,54 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:09,63 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 08:54:09,64 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:09,64 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:09,71 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:54:09,437 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:09,493 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test/user/shardmanager-config_testAddShardReplica/member-1-shard-cars-config_testAddShardReplica#-1453644501] to Actor[pekko://cluster-test/deadLetters] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:09,493 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[pekko://cluster-test/user/shardmanager-oper_testAddShardReplica/member-1-shard-cars-oper_testAddShardReplica#1045737423] to Actor[pekko://cluster-test/deadLetters] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:10,659 AM [cluster-test-pekko.actor.default-dispatcher-32] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2559,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:12,253 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: leader transfer aborted 08:54:12,253 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testAddShardReplica: Failed to transfer leadership in 9.258 s 08:54:12,253 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testAddShardReplica: In unpauseLeader 08:54:12,256 AM [cluster-test-shard-dispatcher-11] [WARN] Leader - member-1-shard-cars-config_testAddShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-config_testAddShardReplica from Leader to IsolatedLeader 08:54:12,257 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:54:12,257 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:12,258 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testAddShardReplica: Stopping Shard 08:54:12,258 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testAddShardReplica: journal closed 08:54:12,258 AM [cluster-test-pekko.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:54:12,259 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:54:12,259 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testAddShardReplica, generation=0} 08:54:12,260 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testAddShardReplica 08:54:12,260 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 08:54:12,260 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c11f656 08:54:12,260 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testAddShardReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c11f656 08:54:12,260 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: pauseLeader successfully completed - doing transfer 08:54:12,260 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3b78451b 08:54:12,312 AM [cluster-test-shard-dispatcher-28] [WARN] Leader - member-1-shard-cars-oper_testAddShardReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 08:54:12,313 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testAddShardReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 1 08:54:12,313 AM [cluster-test-shard-dispatcher-28] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: leader transfer aborted 08:54:12,313 AM [cluster-test-shard-dispatcher-28] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testAddShardReplica: Failed to transfer leadership in 52.81 ms 08:54:12,313 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-1-shard-cars-oper_testAddShardReplica: In unpauseLeader 08:54:12,313 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testAddShardReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:12,313 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testAddShardReplica , received role change from Leader to IsolatedLeader 08:54:12,313 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testAddShardReplica from Leader to IsolatedLeader 08:54:12,313 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testAddShardReplica: Stopping Shard 08:54:12,314 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testAddShardReplica: journal closed 08:54:12,314 AM [cluster-test-pekko.actor.default-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:54:12,314 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:54:12,316 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:12,317 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:12,319 AM [cluster-test-pekko.actor.default-dispatcher-32] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:54:12.321] [cluster-test-pekko.actor.internal-dispatcher-5] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:12,349 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:54:12,368 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [1210060443710313379] 08:54:12,368 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:54:12,371 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:54:12,371 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:54:12,372 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:12,372 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:12,380 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:12,381 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:54:12,382 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:54:12,382 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:54:12,382 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:12,386 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 08:54:12,391 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2140132076] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,393 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:54:12,393 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:54:12,394 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:54:12,395 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,395 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,395 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:12,395 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,395 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,398 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:12,399 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@74fef982} 08:54:12,400 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 08:54:12,400 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:12,400 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,400 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:12,400 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,400 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,400 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-542775482] 08:54:12,400 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,401 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,401 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,401 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-309172328] 08:54:12,401 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-542775482] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,401 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-309172328] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,402 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:54:12,402 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:54:12,402 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:12,402 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#2060155426 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,402 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:54:12,403 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#2060155426] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,403 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-747568961 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,403 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:54:12,404 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:54:12,404 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:54:12,405 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:12,405 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:54:12,406 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-747568961] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,406 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:12,407 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:54:12,407 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:54:12,408 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-954333382] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,409 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:54:12,410 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:54:12,410 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 08:54:12,410 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:54:12,410 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,410 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:54:12,410 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,410 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:12,410 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,411 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,411 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:54:12,411 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 08:54:12,412 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:12,412 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@610bca64} 08:54:12,413 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 08:54:12,413 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:12,413 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,417 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,417 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,417 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:54:12,417 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 08:54:12,417 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:54:12,417 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,417 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,417 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-690208960] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,417 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:54:12,417 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:54:12,417 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#1646243430 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,418 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#1646243430] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,418 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 08:54:12,418 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 08:54:12,418 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:54:12,418 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,418 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,419 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-690208960] 08:54:12,419 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,419 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 08:54:12,419 AM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,419 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,419 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards on shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1995340984] 08:54:12,419 AM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:12,420 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:54:12,420 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1995340984] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,421 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#337265671 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,421 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#337265671] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,421 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:54:12,422 AM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:54:12,423 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:54:12,423 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 08:54:12,423 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,423 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,423 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:54:12,423 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:54:12,423 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 08:54:12,423 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:54:12,425 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:54:12,425 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:12,427 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:54:12,427 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:54:12,427 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 08:54:12,427 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,427 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,427 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:54:12,428 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:54:12,428 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 08:54:12,434 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-3669914284777351352] 08:54:12,435 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 08:54:12,437 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:12,437 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:54:12,438 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:12,438 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:12,451 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:12,453 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:12,460 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 08:54:12,469 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1343702080] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,469 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:54:12,469 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:54:12,471 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:54:12,471 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,471 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,472 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:12,476 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@6022103} 08:54:12,476 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 08:54:12,476 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:12,477 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:12,485 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-config_testAddShardReplica-fe-1-txn-0-0} 08:54:12,486 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:54:12,490 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:12,490 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 08:54:12,501 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:12,501 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,502 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:54:12,504 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-433332619] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,506 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:54:12,508 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,509 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#2093847440] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,509 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:54:12,510 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-503727591 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,510 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#-503727591] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,510 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:54:12,511 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:54:12,512 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-1448182730] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,512 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:54:12,512 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:54:12,512 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#31423954 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,512 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#31423954] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,512 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:54:12,512 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:54:12,514 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:12,514 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:54:12,515 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:12,515 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,515 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,515 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,515 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,516 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:54:12,516 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:54:12,516 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,517 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,517 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,517 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:54:12,517 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,517 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:54:12,517 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 08:54:12,517 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:12,517 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,517 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 08:54:12,517 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 08:54:12,517 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@373d2559} 08:54:12,517 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:12,517 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,517 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,518 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:12,518 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,518 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,518 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,518 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,518 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#125547146] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,518 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:54:12,518 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:54:12,518 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#-331074747] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,519 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-209636957] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,519 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:54:12,519 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#2108157772 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,519 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#2108157772] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,519 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#-209636957 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,519 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:54:12,519 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:54:12,519 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:54:12,519 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 08:54:12,519 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,519 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,519 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:54:12,519 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,519 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 08:54:12,519 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 08:54:12,520 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:54:12,520 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:54:12,520 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:54:12,524 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:12,525 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:12,527 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:54:12,527 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:54:12,527 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 08:54:12,527 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,527 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,527 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:54:12,528 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,528 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:54:12,528 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 08:54:12,528 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,528 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,528 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:54:12,528 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:54:12,528 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:54:12,528 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,528 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 08:54:12,528 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:54:12,528 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 08:54:12,532 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-1343129712248945096] 08:54:12,533 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 08:54:12,535 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:12,535 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:54:12,540 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:12,541 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:12,544 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:12,545 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:12,547 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForAllShards 08:54:12,553 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1708120463] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,557 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:54:12,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:54:12,563 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:54:12,563 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,563 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,565 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:12,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@18864065} 08:54:12,566 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} started 08:54:12,567 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:12,568 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:12,568 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:12,569 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,569 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,570 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:54:12,570 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-830317641] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,570 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:54:12,571 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1206575936 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,572 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:54:12,572 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1996634350 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,572 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards-notifier#1206575936] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,577 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-751112254] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,577 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards-notifier#-1996634350] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,577 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:54:12,579 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:54:12,579 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1058955524] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,580 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:54:12,580 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:54:12,581 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:12,584 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:54:12,584 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:54:12,585 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:54:12,585 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:54:12,585 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards is initialized 08:54:12,586 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:12,588 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:54:12,588 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,589 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,590 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:54:12,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:54:12,591 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards is initialized 08:54:12,591 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:54:12,591 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 08:54:12,591 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards from null to Follower 08:54:12,593 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,593 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,593 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:54:12,595 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:12,595 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards 08:54:12,595 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards from null to Follower 08:54:12,599 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@673626db} 08:54:12,601 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} started 08:54:12,602 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:12,602 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:12,602 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,602 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,603 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:54:12,603 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-318076733] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,604 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:54:12,604 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1842993686] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,604 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards-notifier#1842993686 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,605 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Shard created, persistent : true 08:54:12,605 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Starting recovery with journal batch size 1 08:54:12,606 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-2107822895 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,606 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-853905607] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,606 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:54:12,606 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards-notifier#-2107822895] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:12,607 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal open: applyTo=0 08:54:12,608 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:12,609 AM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:12,611 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:54:12,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:54:12,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards is initialized 08:54:12,612 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:54:12,612 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:54:12,612 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:54:12,613 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards from null to Follower 08:54:12,614 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:54:12,614 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:12,615 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,615 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,615 AM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Local TermInfo store seeded with TermInfo{term=0} 08:54:12,615 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:12,615 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,615 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,615 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,615 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,615 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:12,615 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:12,615 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:12,615 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,615 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,615 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,615 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,615 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:54:12,615 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:12,615 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards is initialized 08:54:12,615 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from null to Follower 08:54:12,615 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards 08:54:12,616 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards from null to Follower 08:54:12,750 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 08:54:12,750 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:12,750 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,750 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,751 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 08:54:12,751 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 08:54:12,799 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 08:54:12,799 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:12,799 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,799 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,799 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 08:54:12,799 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Follower to Candidate 08:54:12,830 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 08:54:12,831 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:12,831 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,831 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,831 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 08:54:12,831 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 08:54:12,851 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 1 08:54:12,851 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:12,851 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: false, hasLeader : false 08:54:12,851 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:12,851 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Follower to Candidate 08:54:12,851 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Follower to Candidate 08:54:13,159 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 08:54:13,179 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 08:54:13,209 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 08:54:13,219 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 2 08:54:13,389 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:54:13,389 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:54:13,389 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:13,389 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,389 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,389 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:13,389 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,389 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,389 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,389 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,389 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:13,389 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,389 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,389 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:13,390 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,390 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,397 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:13,474 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:13,474 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:13,474 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:13,474 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,474 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,475 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,475 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,475 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,475 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,475 AM [cluster-test-shard-dispatcher-23] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatesForAllShards set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,539 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 08:54:13,541 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:13,541 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:13,549 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:54:13,549 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:54:13,549 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 08:54:13,549 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 08:54:13,550 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6fcc2835 08:54:13,550 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 08:54:13,550 AM [cluster-test-shard-dispatcher-22] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 08:54:13,551 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 08:54:13,551 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1903f6b1 08:54:13,551 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done false 08:54:13,551 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6916c528 08:54:13,552 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:13,552 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:13,554 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:54:13,557 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:54:13,557 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:54:13,557 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 08:54:13,557 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@37357fb8 08:54:13,557 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 08:54:13,557 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 08:54:13,558 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1a2710d7 08:54:13,558 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 08:54:13,558 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done false 08:54:13,558 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5c017bf5 08:54:13,579 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 08:54:13,580 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:13,580 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:13,587 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:54:13,587 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:54:13,587 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 08:54:13,587 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 08:54:13,587 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2550e3e8 08:54:13,587 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards from Candidate to Leader 08:54:13,587 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 08:54:13,588 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 08:54:13,588 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done false 08:54:13,588 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@135bf021 08:54:13,588 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 08:54:13,588 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@520cc2c3 08:54:13,588 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatesForAllShards is ready 08:54:13,589 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate): Starting new election term 3 08:54:13,590 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:13,591 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:13,596 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:54:13,596 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: onLeadershipChange, isLeader: true, hasLeader : true 08:54:13,596 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: became leader with frontend state for [] 08:54:13,596 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards , received role change from Candidate to Leader 08:54:13,596 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6bf3a114 08:54:13,596 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards from Candidate to Leader 08:54:13,596 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 08:54:13,596 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 08:54:13,596 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:13,596 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 08:54:13,596 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:13,597 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 08:54:13,597 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1d160d11 08:54:13,597 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1eedd533 08:54:13,597 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 08:54:13,597 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatesForAllShards is ready 08:54:13,597 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done false 08:54:13,597 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 08:54:13,597 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:13,597 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} is now ready 08:54:13,597 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2140132076] 08:54:13,598 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:13,598 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-542775482] 08:54:13,598 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-542775482] 08:54:13,598 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1343702080] 08:54:13,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:54:13,598 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#2093847440] 08:54:13,598 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#2093847440] 08:54:13,599 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1708120463] 08:54:13,599 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:54:13,599 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-830317641] 08:54:13,599 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-830317641] 08:54:13,605 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for all shards 08:54:13,605 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testChangeMemberVotingStatesForAllShards 08:54:13,606 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testChangeMemberVotingStatesForAllShards 08:54:13,606 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 08:54:13,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 08:54:13,607 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:54:13,607 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 08:54:13,607 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,607 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:54:13,607 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=people, meberVotingStatusMap={member-2=false, member-3=false}] 08:54:13,608 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:54:13,608 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:54:13,608 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards=false, member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,608 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-config_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,608 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards=false, member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,610 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:54:13,612 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,612 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard cars 08:54:13,612 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:54:13,613 AM [main] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:54:13,613 AM [main] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 08:54:13,613 AM [cluster-test-shard-dispatcher-22] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-3-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:54:13,613 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: VotingConfig{serverInfo=[ServerInfo[peerId=member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards, isVoting=true]]} has been successfully replicated to a majority of followers 08:54:13,614 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,614 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 08:54:13,614 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:54:13,614 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 08:54:13,614 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,615 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard people 08:54:13,615 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:54:13,615 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 08:54:13,621 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 08:54:13,622 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards status sync done true 08:54:13,622 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards} from local shard pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,622 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: ChangeServersVotingStatus succeeded for shard people 08:54:13,622 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:54:13,622 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 08:54:13,624 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2140132076] 08:54:13,624 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:13,624 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-542775482] 08:54:13,624 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards#-542775482] 08:54:13,624 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 08:54:13,624 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 08:54:13,624 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 08:54:13,624 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards status sync done true 08:54:13,625 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatesForAllShards status sync done true 08:54:13,625 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#2140132076] 08:54:13,625 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:54:13,625 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards status sync done true 08:54:13,625 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-309172328] 08:54:13,628 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-1-shard-people-config_testChangeMemberVotingStatesForAllShards#-309172328] 08:54:13,629 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-954333382] 08:54:13,629 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:54:13,629 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-690208960] 08:54:13,629 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-690208960] 08:54:13,629 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-954333382] 08:54:13,629 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:54:13,629 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1995340984] 08:54:13,630 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards#-1995340984] 08:54:13,630 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1343702080] 08:54:13,630 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:54:13,631 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#2093847440] 08:54:13,631 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards#2093847440] 08:54:13,631 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#1343702080] 08:54:13,631 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:54:13,631 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-1448182730] 08:54:13,631 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-2-shard-people-config_testChangeMemberVotingStatesForAllShards#-1448182730] 08:54:13,631 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-433332619] 08:54:13,631 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:54:13,631 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#125547146] 08:54:13,631 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards#125547146] 08:54:13,631 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#-433332619] 08:54:13,631 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 08:54:13,631 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#-331074747] 08:54:13,631 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards#-331074747] 08:54:13,631 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1708120463] 08:54:13,632 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:54:13,632 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-830317641] 08:54:13,632 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards#-830317641] 08:54:13,632 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards#-1708120463] 08:54:13,632 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:54:13,632 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-751112254] 08:54:13,632 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForAllShards/member-3-shard-people-config_testChangeMemberVotingStatesForAllShards#-751112254] 08:54:13,632 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1058955524] 08:54:13,632 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:54:13,632 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-853905607] 08:54:13,632 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards#-853905607] 08:54:13,632 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards#1058955524] 08:54:13,632 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:54:13,632 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-318076733] 08:54:13,632 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForAllShards/member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards#-318076733] 08:54:13,633 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:13,633 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,634 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,634 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 08:54:13,634 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:54:13,634 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:54:13,635 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 08:54:13,635 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 08:54:13,635 AM [cluster-test-pekko.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:54:13,635 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:54:13,635 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:13,636 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,636 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,636 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:54:13,636 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:54:13,636 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:54:13,637 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 08:54:13,637 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 08:54:13,637 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:54:13,637 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:54:13,641 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:13,641 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:13,645 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:13,646 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:13,646 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:13,646 AM [cluster-test-pekko.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:13,649 AM [cluster-test-pekko.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:54:13,665 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:13,666 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:13,666 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:13,666 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:54:13,667 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:54:13,667 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:54:13,667 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 08:54:13,667 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 08:54:13,668 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:54:13,669 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:54:13,669 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:13,669 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:13,670 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:13,670 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:54:13,670 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:54:13,670 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:54:13,670 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 08:54:13,671 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 08:54:13,671 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:54:13,671 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:54:13,672 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:13,672 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:14,407 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:54:14,657 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-1343129712248945096] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:14,688 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-3669914284777351352] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:14,688 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:14,689 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:14,691 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:54:14.693] [cluster-test-pekko.actor.internal-dispatcher-5] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:14,697 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:14,697 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards 08:54:14,698 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForAllShards 08:54:14,698 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:54:14,698 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6998632e 08:54:14,698 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6998632e 08:54:14,698 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 08:54:14,698 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@120dc97c 08:54:14,698 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@120dc97c 08:54:14,698 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 08:54:14,698 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ededd41 08:54:14,698 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6b471da1 08:54:15,147 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 08:54:15,147 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 448.9 ms 08:54:15,147 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 08:54:15,148 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:54:15,149 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForAllShards: journal closed 08:54:15,187 AM [cluster-test-shard-dispatcher-22] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: leader transfer aborted 08:54:15,187 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 489.6 ms 08:54:15,188 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: In unpauseLeader 08:54:15,188 AM [cluster-test-shard-dispatcher-22] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:54:15,189 AM [cluster-test-shard-dispatcher-22] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForAllShards: journal closed 08:54:15,189 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:54:15,189 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:54:15,190 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForAllShards, generation=0} 08:54:15,190 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards 08:54:15,190 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards 08:54:15,190 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:54:15,190 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@65ab32e4 08:54:15,191 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5964188e 08:54:15,191 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a88cae 08:54:15,191 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3a88cae 08:54:15,191 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 08:54:15,191 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2b970932 08:54:15,191 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2b970932 08:54:15,191 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: pauseLeader successfully completed - doing transfer 08:54:15,506 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 08:54:15,506 AM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 316.2 ms 08:54:15,507 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 08:54:15,507 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:54:15,508 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForAllShards: journal closed 08:54:15,797 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: leader transfer aborted 08:54:15,797 AM [cluster-test-shard-dispatcher-12] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Failed to transfer leadership in 606.9 ms 08:54:15,798 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: In unpauseLeader 08:54:15,798 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: Stopping Shard 08:54:15,799 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForAllShards: journal closed 08:54:15,799 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:54:15,799 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:54:15,801 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:15,801 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:15,803 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:54:15.806] [cluster-test-pekko.actor.internal-dispatcher-3] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:15,833 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:54:15,843 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [3055173251770814720] 08:54:15,843 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:54:15,843 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:54:15,843 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:54:15,846 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:15,847 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:15,853 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:15,855 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:15,856 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:54:15,856 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:54:15,856 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:54:15,857 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testGetPrefixShardRole 08:54:15,862 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:54:15,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:54:15,863 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1343852323] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:15,865 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:54:15,865 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-default-config_testGetPrefixShardRole 08:54:15,865 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:15,865 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 08:54:15,868 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 08:54:15,869 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@792de0fc} 08:54:15,869 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} started 08:54:15,870 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:15,871 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:15,873 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:15,873 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-default-config_testGetPrefixShardRole 08:54:15,874 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testGetPrefixShardRole 08:54:15,878 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 08:54:15,878 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-config_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole 08:54:15,878 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-default-config_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole on shard member-1-shard-default-config_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#17110604] 08:54:15,878 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Shard created, persistent : true 08:54:15,879 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-default-config_testGetPrefixShardRole: Starting recovery with journal batch size 1 08:54:15,879 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#17110604] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:15,879 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-1275482935 created and ready for shard:member-1-shard-default-config_testGetPrefixShardRole 08:54:15,880 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole-notifier#-1275482935] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:15,880 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-default-config_testGetPrefixShardRole: journal open: applyTo=0 08:54:15,881 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#-307460307] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:15,881 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:54:15,881 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:54:15,885 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:54:15,885 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-default-oper_testGetPrefixShardRole 08:54:15,886 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:15,886 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 08:54:15,886 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-default-config_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:15,893 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-default-config_testGetPrefixShardRole: Local TermInfo store seeded with TermInfo{term=0} 08:54:15,893 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testGetPrefixShardRole: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 08:54:15,893 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 08:54:15,893 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:15,893 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from null to Follower 08:54:15,893 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [default] 08:54:15,893 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-default-config_testGetPrefixShardRole is initialized 08:54:15,893 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@218a532e} 08:54:15,894 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} started 08:54:15,894 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:15,894 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-default-oper_testGetPrefixShardRole 08:54:15,895 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Shard created, persistent : true 08:54:15,895 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-438957558] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:15,896 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#706219409 created and ready for shard:member-1-shard-default-oper_testGetPrefixShardRole 08:54:15,896 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole-notifier#706219409] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:15,896 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-default-oper_testGetPrefixShardRole: Starting recovery with journal batch size 1 08:54:15,896 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 08:54:15,896 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-oper_testGetPrefixShardRole with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole to pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole 08:54:15,897 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-default-oper_testGetPrefixShardRole on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole on shard member-1-shard-default-oper_testGetPrefixShardRole whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole/member-1-shard-default-oper_testGetPrefixShardRole#-438957558] 08:54:15,898 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-default-oper_testGetPrefixShardRole: journal open: applyTo=0 08:54:15,899 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-default-oper_testGetPrefixShardRole: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:15,903 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-default-config_testGetPrefixShardRole (Candidate): Starting new election term 1 08:54:15,903 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-default-config_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:15,903 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 08:54:15,903 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:15,903 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Follower to Candidate 08:54:15,903 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-default-config_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:54:15,903 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-config_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 08:54:15,903 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: became leader with frontend state for [] 08:54:15,903 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , received role change from Candidate to Leader 08:54:15,903 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-config_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole 08:54:15,903 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@65e527d7 08:54:15,903 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-default-config_testGetPrefixShardRole from Candidate to Leader 08:54:15,904 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testGetPrefixShardRole is ready 08:54:15,904 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-default-oper_testGetPrefixShardRole: Local TermInfo store seeded with TermInfo{term=0} 08:54:15,905 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 08:54:15,905 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:15,905 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [default] 08:54:15,905 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-default-oper_testGetPrefixShardRole is initialized 08:54:15,905 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from null to Follower 08:54:15,907 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-default-oper_testGetPrefixShardRole (Candidate): Starting new election term 1 08:54:15,907 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-default-oper_testGetPrefixShardRole (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:15,907 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: false, hasLeader : false 08:54:15,907 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:15,907 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Follower to Candidate 08:54:15,907 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-default-oper_testGetPrefixShardRole (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:54:15,907 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-oper_testGetPrefixShardRole: onLeadershipChange, isLeader: true, hasLeader : true 08:54:15,907 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: became leader with frontend state for [] 08:54:15,907 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , received role change from Candidate to Leader 08:54:15,907 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-oper_testGetPrefixShardRole , registered listener pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole 08:54:15,907 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@61e1cd0e 08:54:15,908 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-default-oper_testGetPrefixShardRole from Candidate to Leader 08:54:15,908 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testGetPrefixShardRole is ready 08:54:15,944 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1343852323] 08:54:15,944 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : default 08:54:15,944 AM [main] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole/member-1-shard-default-config_testGetPrefixShardRole#17110604] 08:54:15,945 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testGetPrefixShardRole, generation=0} 08:54:15,946 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-default-config_testGetPrefixShardRole 08:54:15,946 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 08:54:15,946 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-default-config_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@7d87ee89 08:54:15,946 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-default-config_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@7d87ee89 08:54:15,946 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-default-config_testGetPrefixShardRole: Stopping Shard 08:54:15,947 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-default-config_testGetPrefixShardRole: journal closed 08:54:15,947 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:54:15,947 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:54:15,948 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testGetPrefixShardRole, generation=0} 08:54:15,948 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1343852323] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1343852323] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:15,948 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-default-oper_testGetPrefixShardRole 08:54:15,948 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 1 shards 08:54:15,948 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-default-oper_testGetPrefixShardRole: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@20e7a50f 08:54:15,948 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-default-oper_testGetPrefixShardRole: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@20e7a50f 08:54:15,948 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1343852323] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-config_testGetPrefixShardRole#-1343852323] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:15,948 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda/0x00007fccc082b6c8] to Actor[pekko://cluster-test/user/$a#1439723835] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/$a#1439723835] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:15,949 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-default-oper_testGetPrefixShardRole: Stopping Shard 08:54:15,949 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-default-oper_testGetPrefixShardRole: journal closed 08:54:15,950 AM [cluster-test-pekko.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:54:15,950 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:54:15,950 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RepointableActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#-307460307] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/shardmanager-oper_testGetPrefixShardRole#-307460307] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:15,951 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:15,951 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:15,960 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:54:15,989 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:54:16,16 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [4514367358688025530] 08:54:16,17 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:54:16,25 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:54:16,25 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:54:16,27 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:16,27 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:16,36 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:16,37 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:54:16,37 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:54:16,37 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:16,37 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:54:16,41 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,48 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1500179338] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,49 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:54:16,50 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 08:54:16,52 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:16,53 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:16,54 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:54:16,54 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,55 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:54:16,56 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#2040066976] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,57 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:54:16,57 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:54:16,57 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:54:16,57 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,58 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,58 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:16,58 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,58 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,58 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@5a391263} 08:54:16,59 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:16,59 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,59 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,59 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,59 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1084315598] 08:54:16,59 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,60 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,60 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,60 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1345234154] 08:54:16,60 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:54:16,60 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:54:16,61 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1345234154] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,62 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1259012542 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,62 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:54:16,62 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1117745402 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,63 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:54:16,63 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 08:54:16,64 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1084315598] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,64 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1259012542] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,64 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1117745402] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,65 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:54:16,66 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:54:16,67 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:54:16,67 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,68 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,68 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:16,68 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,68 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,68 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@64ca78ce} 08:54:16,68 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:16,68 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,68 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:54:16,69 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:54:16,69 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1123225109] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,70 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,70 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#925201733 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,70 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,70 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1123225109] 08:54:16,70 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,70 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#925201733] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,70 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:54:16,71 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:16,71 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:54:16,71 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:16,73 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:54:16,73 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 08:54:16,73 AM [cluster-test-shard-dispatcher-16] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 08:54:16,73 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 08:54:16,74 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 08:54:16,74 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 08:54:16,74 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 08:54:16,74 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:54:16,75 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:54:16,75 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-346431762] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,75 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1615332595 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,76 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-346431762] 08:54:16,76 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1615332595] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,77 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:16,77 AM [cluster-test-shard-dispatcher-16] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 4.080 ms, resetting timer for the next recovery snapshot 08:54:16,77 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:54:16,78 AM [cluster-test-shard-dispatcher-18] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 3.362 ms, resetting timer for the next recovery snapshot 08:54:16,81 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:16,82 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 08:54:16,82 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 08:54:16,82 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 08:54:16,83 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:54:16,83 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:16,83 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:54:16,83 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:16,83 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:16,84 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:16,84 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:54:16,84 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:54:16,84 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:54:16,84 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:54:16,84 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,84 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 08:54:16,84 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 08:54:16,84 AM [cluster-test-shard-dispatcher-17] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 08:54:16,84 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:54:16,84 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,84 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:54:16,86 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 3.128 ms, resetting timer for the next recovery snapshot 08:54:16,86 AM [cluster-test-shard-dispatcher-17] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.257 ms, resetting timer for the next recovery snapshot 08:54:16,90 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:16,90 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:16,90 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:54:16,90 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:54:16,90 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:54:16,90 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,91 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:54:16,91 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:54:16,91 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:54:16,91 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:16,91 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:16,91 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:54:16,91 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,91 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:54:16,93 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-2568902701337861950] 08:54:16,94 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 08:54:16,96 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:16,96 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:54:16,97 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:16,97 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:16,108 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:16,110 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:16,112 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,118 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:54:16,119 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:54:16,120 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1792549770] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,124 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:54:16,124 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,124 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,128 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:54:16,129 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@58646fb2} 08:54:16,129 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 08:54:16,129 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:16,131 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:16,132 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:16,132 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,133 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,134 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,135 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1677651585] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,135 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:54:16,135 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1423244501 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,135 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1423244501] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,137 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#347164816] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,138 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:54:16,138 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:54:16,138 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:54:16,139 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1575605562 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,139 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#568261203] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,139 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:54:16,139 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1575605562] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,140 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:54:16,140 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:54:16,140 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:54:16,141 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:54:16,141 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,141 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,145 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:54:16,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@1d66fada} 08:54:16,154 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:16,155 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:16,155 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 08:54:16,156 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:16,156 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,157 AM [cluster-test-shard-dispatcher-23] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:54:16,157 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1776433952] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,158 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-876108003 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,158 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:54:16,158 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-876108003] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,160 AM [cluster-test-shard-dispatcher-23] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:54:16,160 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 08:54:16,160 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 08:54:16,160 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 08:54:16,164 AM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:16,164 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 08:54:16,164 AM [cluster-test-shard-dispatcher-26] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 08:54:16,164 AM [cluster-test-shard-dispatcher-26] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 08:54:16,165 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,166 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 5.481 ms, resetting timer for the next recovery snapshot 08:54:16,166 AM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 08:54:16,166 AM [cluster-test-shard-dispatcher-23] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 08:54:16,166 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 08:54:16,169 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:54:16,169 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1236839265] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,169 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:54:16,169 AM [cluster-test-pekko.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1022044116 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,169 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#1022044116] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,171 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:54:16,172 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:16,172 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:16,172 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:54:16,172 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:54:16,172 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:54:16,173 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,173 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:54:16,173 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:16,174 AM [cluster-test-shard-dispatcher-23] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 7.734 ms, resetting timer for the next recovery snapshot 08:54:16,174 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 08:54:16,174 AM [cluster-test-shard-dispatcher-24] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 08:54:16,174 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 08:54:16,176 AM [cluster-test-shard-dispatcher-26] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 11.92 ms, resetting timer for the next recovery snapshot 08:54:16,178 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:54:16,180 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:16,180 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:16,180 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:54:16,180 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:54:16,180 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:54:16,180 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,180 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:54:16,180 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:54:16,180 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:54:16,180 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:16,180 AM [cluster-test-shard-dispatcher-26] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:16,180 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:54:16,181 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,181 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:54:16,181 AM [cluster-test-shard-dispatcher-24] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 6.412 ms, resetting timer for the next recovery snapshot 08:54:16,184 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:16,184 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:16,184 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:54:16,184 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:54:16,184 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:54:16,184 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,184 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:54:16,200 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-3274882572082161070] 08:54:16,202 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 08:54:16,204 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:16,204 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:54:16,205 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:16,208 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:54:16,210 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:16,211 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,211 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,221 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:16,224 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:16,225 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,231 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:54:16,231 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:54:16,238 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1542499550] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,242 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:54:16,242 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,243 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,245 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:54:16,245 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@fb684ec} 08:54:16,245 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 08:54:16,246 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:16,247 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:16,249 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,250 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:16,250 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,251 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#2029662189] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,251 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:54:16,251 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:54:16,251 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#2119272031] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,251 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:54:16,252 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:54:16,252 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1763259943 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,252 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-1763259943] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,253 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:54:16,254 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,254 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:54:16,255 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,256 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,256 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:54:16,256 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#137045212 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,256 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:54:16,257 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:54:16,258 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1781520643] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,258 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader-notifier#137045212] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,259 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:54:16,259 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:54:16,259 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@6e8a0306} 08:54:16,259 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} started 08:54:16,260 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:16,260 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,260 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,260 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:54:16,261 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1150430201] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,261 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-377343807 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,265 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Shard created, persistent : true 08:54:16,273 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:16,273 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:54:16,273 AM [cluster-test-shard-dispatcher-31] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Starting recovery with journal batch size 1 08:54:16,274 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-919730616 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,274 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-377343807] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,274 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:54:16,274 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-728113000] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,274 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader-notifier#-919730616] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:16,276 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 08:54:16,276 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:16,276 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 08:54:16,276 AM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 08:54:16,276 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,276 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,276 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,276 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,276 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:16,276 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,276 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,276 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,276 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,277 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:54:16,278 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal open: applyTo=0 08:54:16,279 AM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.906 ms, resetting timer for the next recovery snapshot 08:54:16,280 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:16,281 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:16,281 AM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:16,281 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:54:16,281 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:54:16,281 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:16,282 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:16,282 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:54:16,282 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,282 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,282 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:54:16,284 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 08:54:16,284 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 08:54:16,284 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 08:54:16,284 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 08:54:16,284 AM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 08:54:16,284 AM [cluster-test-shard-dispatcher-31] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 08:54:16,285 AM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Local TermInfo store seeded with TermInfo{term=1} 08:54:16,285 AM [cluster-test-shard-dispatcher-34] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: taking snapshot to clear Pekko persistence to 2 08:54:16,285 AM [cluster-test-shard-dispatcher-34] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Taking snapshot on entry with index -1 08:54:16,286 AM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.360 ms, resetting timer for the next recovery snapshot 08:54:16,287 AM [cluster-test-shard-dispatcher-31] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 3.069 ms, resetting timer for the next recovery snapshot 08:54:16,287 AM [cluster-test-shard-dispatcher-34] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Snapshot completed in 2.013 ms, resetting timer for the next recovery snapshot 08:54:16,291 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:54:16,291 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:54:16,291 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:16,291 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:16,291 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:54:16,291 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,291 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,291 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:54:16,293 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:54:16,293 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:54:16,293 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:54:16,293 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader is initialized 08:54:16,293 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:16,293 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:16,293 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:16,293 AM [cluster-test-shard-dispatcher-34] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:16,293 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:54:16,293 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from null to Follower 08:54:16,293 AM [cluster-test-shard-dispatcher-34] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,293 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,293 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,293 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:16,293 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:54:16,293 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from null to Follower 08:54:17,45 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:54:17,45 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:17,46 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-16] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,46 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:17,54 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:17,54 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-26] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,54 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,61 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1500179338] 08:54:17,61 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:17,61 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1084315598] 08:54:17,61 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1084315598] 08:54:17,62 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1500179338] 08:54:17,62 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:17,62 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1084315598] 08:54:17,62 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1084315598] 08:54:17,62 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 08:54:17,62 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,63 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,63 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:54:17,63 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:54:17,63 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:54:17,63 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:54:17,63 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:54:17,63 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:54:17,64 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:17,64 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:54:17,64 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,64 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,64 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,65 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true, member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=false, member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,72 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 08:54:17,72 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 08:54:17,72 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:54:17,72 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:54:17,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:17,72 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:17,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:17,72 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:17,72 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 08:54:17,72 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 08:54:17,73 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 08:54:17,73 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:54:17,73 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:17,73 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 08:54:17,73 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:17,73 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 08:54:17,73 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 08:54:17,73 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 08:54:17,73 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): Starting new election term 2 08:54:17,73 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:54:17,74 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: false, hasLeader : false 08:54:17,74 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:17,74 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Follower to Candidate 08:54:17,74 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Follower to Candidate 08:54:17,74 AM [cluster-test-shard-dispatcher-24] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:17,74 AM [cluster-test-shard-dispatcher-11] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:17,75 AM [cluster-test-shard-dispatcher-26] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:17,75 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:17,75 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:17,75 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:17,75 AM [cluster-test-shard-dispatcher-34] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:17,75 AM [cluster-test-shard-dispatcher-23] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Follower): Term 2 in "RequestVote{term=2, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:17,80 AM [cluster-test-shard-dispatcher-16] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:54:17,81 AM [cluster-test-shard-dispatcher-16] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:54:17,81 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:54:17,81 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 08:54:17,81 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:54:17,82 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2e048dc0 08:54:17,82 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@520153a6 08:54:17,82 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:54:17,82 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@eac5a64 08:54:17,82 AM [cluster-test-shard-dispatcher-17] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:54:17,82 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:54:17,82 AM [cluster-test-shard-dispatcher-15] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:54:17,83 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:54:17,83 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:54:17,83 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:54:17,83 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:54:17,83 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 08:54:17,83 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 08:54:17,83 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 2 08:54:17,83 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:54:17,83 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: became leader with frontend state for [] 08:54:17,83 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 08:54:17,83 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@45d0d960 08:54:17,83 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,83 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 08:54:17,83 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@75a47392 08:54:17,83 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 08:54:17,83 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:54:17,83 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 08:54:17,84 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 08:54:17,84 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 08:54:17,84 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,84 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 08:54:17,84 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:54:17,84 AM [cluster-test-pekko.actor.default-dispatcher-20] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 08:54:17,85 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:54:17,85 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2ca62005 08:54:17,85 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 08:54:17,85 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4fd9ae27 08:54:17,85 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:54:17,85 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:54:17,85 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:54:17,85 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:54:17,85 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:54:17,85 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@63636f74 08:54:17,85 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 08:54:17,86 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:54:17,86 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:54:17,86 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:54:17,86 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,86 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 08:54:17,86 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:54:17,86 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 08:54:17,87 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done false 08:54:17,87 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:54:17,87 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3bec5934 08:54:17,87 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 08:54:17,88 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:54:17,88 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 08:54:17,89 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 08:54:17,89 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:54:17,92 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-4-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 08:54:17,93 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-6-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 08:54:17,93 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:54:17,93 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:54:17,93 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 08:54:17,93 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 08:54:17,94 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-6-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-4-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-5-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 08:54:17,94 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: VotingConfig{serverInfo=[ServerInfo[peerId=member-5-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-4-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true], ServerInfo[peerId=member-6-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=false], ServerInfo[peerId=member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader, isVoting=true]]} has been successfully replicated to a majority of followers 08:54:17,94 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:54:17,94 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:54:17,94 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:54:17,94 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:54:17,94 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 08:54:17,94 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 08:54:17,96 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 2 08:54:17,96 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: onLeadershipChange, isLeader: true, hasLeader : true 08:54:17,96 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 08:54:17,96 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 08:54:17,96 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithNoInitialLeader is ready 08:54:17,98 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,98 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 08:54:17,98 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:54:17,98 AM [cluster-test-pekko.actor.default-dispatcher-14] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 08:54:17,98 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from Candidate to PreLeader 08:54:17,98 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader , received role change from PreLeader to Leader 08:54:17,98 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@27e7bb5b 08:54:17,98 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from Candidate to PreLeader 08:54:17,98 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader from PreLeader to Leader 08:54:17,98 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 08:54:17,98 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1500179338] 08:54:17,98 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:17,98 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1084315598] 08:54:17,98 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1084315598] 08:54:17,99 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1500179338] 08:54:17,99 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:54:17,99 AM [cluster-test-pekko.actor.default-dispatcher-35] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1345234154] 08:54:17,99 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1345234154] 08:54:17,99 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#2040066976] 08:54:17,99 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:54:17,99 AM [cluster-test-pekko.actor.default-dispatcher-35] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1123225109] 08:54:17,99 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1123225109] 08:54:17,99 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#2040066976] 08:54:17,100 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:54:17,100 AM [cluster-test-pekko.actor.default-dispatcher-35] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-346431762] 08:54:17,100 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-346431762] 08:54:17,100 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1792549770] 08:54:17,103 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@42191f68 08:54:17,103 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6e4dc134 08:54:17,103 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithNoInitialLeader is ready 08:54:17,103 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader status sync done true 08:54:17,103 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:54:17,103 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1677651585] 08:54:17,103 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#-1677651585] 08:54:17,103 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1792549770] 08:54:17,103 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:54:17,103 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#568261203] 08:54:17,103 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#568261203] 08:54:17,103 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#347164816] 08:54:17,103 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:54:17,103 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1776433952] 08:54:17,104 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1776433952] 08:54:17,104 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#347164816] 08:54:17,104 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 08:54:17,104 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1236839265] 08:54:17,104 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-1236839265] 08:54:17,104 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1542499550] 08:54:17,104 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:54:17,104 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#2119272031] 08:54:17,104 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#2119272031] 08:54:17,104 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1542499550] 08:54:17,104 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:54:17,104 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1781520643] 08:54:17,104 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader#1781520643] 08:54:17,105 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#2029662189] 08:54:17,105 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:54:17,105 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1150430201] 08:54:17,105 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1150430201] 08:54:17,105 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#2029662189] 08:54:17,105 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:54:17,105 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-728113000] 08:54:17,105 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader#-728113000] 08:54:17,105 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader#-1500179338] 08:54:17,105 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:17,105 AM [cluster-test-pekko.actor.default-dispatcher-35] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1084315598] 08:54:17,105 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader#1084315598] 08:54:17,106 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader#2040066976] 08:54:17,106 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:54:17,106 AM [cluster-test-pekko.actor.default-dispatcher-35] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1123225109] 08:54:17,106 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithNoInitialLeader/member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader#1123225109] 08:54:17,107 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:54:17,107 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,107 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,107 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 08:54:17,107 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:54:17,108 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:54:17,109 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:54:17,112 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:54:17,113 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:54:17,113 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:54:17,114 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:54:17,114 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,115 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,115 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:54:17,115 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:54:17,115 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:54:17,116 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:54:17,116 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:54:17,117 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:54:17,117 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:54:17,119 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:17,119 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:17,126 AM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:17,126 AM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:17,126 AM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:17,126 AM [cluster-test-pekko.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:17,128 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:54:17,142 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:54:17,143 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,143 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,143 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:54:17,144 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:54:17,144 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:54:17,144 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:54:17,144 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:54:17,144 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:54:17,145 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:54:17,145 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:54:17,146 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,146 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:17,146 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:54:17,146 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:54:17,146 AM [cluster-test-shard-dispatcher-26] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:54:17,146 AM [cluster-test-shard-dispatcher-26] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:54:17,146 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:54:17,147 AM [cluster-test-pekko.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:54:17,147 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:54:17,148 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:17,149 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:18,64 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:54:18,134 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-3274882572082161070] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:18,164 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-2568902701337861950] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:18,170 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:18,170 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:18,173 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:54:18.175] [cluster-test-pekko.actor.internal-dispatcher-9] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:18,187 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:54:18,188 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:18,188 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader 08:54:18,188 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:54:18,189 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4ecc7207 08:54:18,189 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7b00d4c1 08:54:18,189 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7b00d4c1 08:54:18,189 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 08:54:18,189 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6b16726e 08:54:18,189 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5c076c67 08:54:18,189 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5c076c67 08:54:18,189 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 08:54:18,684 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 08:54:18,684 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 495.8 ms 08:54:18,685 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 08:54:18,685 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:54:18,686 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:54:18,694 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 08:54:18,694 AM [cluster-test-shard-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 506.1 ms 08:54:18,695 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 08:54:18,695 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:54:18,696 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:54:18,696 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:54:18,696 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:54:18,697 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithNoInitialLeader, generation=0} 08:54:18,697 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:18,697 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader 08:54:18,697 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:54:18,697 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@284ebdb5 08:54:18,697 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c961a4b 08:54:18,698 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c961a4b 08:54:18,698 AM [cluster-test-shard-dispatcher-17] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 08:54:18,698 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@98b7428 08:54:18,698 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1968243c 08:54:18,698 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1968243c 08:54:18,698 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: pauseLeader successfully completed - doing transfer 08:54:19,14 AM [cluster-test-shard-dispatcher-16] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 08:54:19,14 AM [cluster-test-shard-dispatcher-16] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 316.2 ms 08:54:19,14 AM [cluster-test-shard-dispatcher-16] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 08:54:19,14 AM [cluster-test-shard-dispatcher-15] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: leader transfer aborted 08:54:19,14 AM [cluster-test-shard-dispatcher-15] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Failed to transfer leadership in 316.7 ms 08:54:19,14 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: In unpauseLeader 08:54:19,14 AM [cluster-test-shard-dispatcher-16] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:54:19,15 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: Stopping Shard 08:54:19,15 AM [cluster-test-shard-dispatcher-16] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:54:19,15 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithNoInitialLeader: journal closed 08:54:19,16 AM [cluster-test-pekko.actor.default-dispatcher-34] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:54:19,16 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:54:19,17 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:19,17 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:19,19 AM [cluster-test-pekko.actor.default-dispatcher-34] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:54:19.021] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:19,40 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:54:19,50 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [1877656615508653703] 08:54:19,50 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:54:19,52 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:54:19,53 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:54:19,57 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:19,57 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:19,64 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:19,65 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:19,65 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:54:19,65 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:54:19,65 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:54:19,67 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,70 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1008487736] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,71 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:54:19,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:54:19,76 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:54:19,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,76 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,76 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:19,76 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,76 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,78 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:19,79 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@3b736bec} 08:54:19,79 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 08:54:19,79 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:19,80 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:19,80 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,81 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,81 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,81 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-711227713] 08:54:19,81 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,81 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,81 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,81 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1539810658] 08:54:19,81 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:19,81 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:54:19,82 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1342002474 created and ready for shard:member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,82 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:54:19,82 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-711227713] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,82 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1342002474] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,82 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:54:19,83 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1539810658] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,83 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:54:19,83 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,84 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:54:19,84 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-114967938 created and ready for shard:member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,84 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-114967938] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,85 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1059607326] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,85 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:54:19,85 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:54:19,86 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:54:19,90 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:19,90 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:54:19,90 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,91 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,91 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:19,91 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:19,91 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,92 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,93 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:19,94 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 08:54:19,94 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 08:54:19,94 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 08:54:19,94 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 08:54:19,94 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 08:54:19,94 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 08:54:19,95 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@3d3edada} 08:54:19,95 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 08:54:19,95 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:19,95 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,96 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,96 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,96 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1450511830] 08:54:19,96 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,96 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,96 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,96 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1682178522] 08:54:19,96 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 2.348 ms, resetting timer for the next recovery snapshot 08:54:19,97 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:54:19,97 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:54:19,99 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1450511830] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,99 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:54:19,99 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-381871178 created and ready for shard:member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,99 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-381871178] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,99 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:54:19,99 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1682178522] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,99 AM [cluster-test-shard-dispatcher-12] [INFO] Recovery - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 5.310 ms, resetting timer for the next recovery snapshot 08:54:19,99 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:54:19,100 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#2079896394 created and ready for shard:member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,100 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#2079896394] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,100 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:54:19,100 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:54:19,101 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:54:19,101 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:19,102 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:54:19,102 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:54:19,102 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,102 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,102 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:54:19,102 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,103 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:54:19,103 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 08:54:19,103 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 08:54:19,103 AM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 08:54:19,104 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,104 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,104 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:54:19,104 AM [cluster-test-pekko.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,104 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:54:19,105 AM [cluster-test-shard-dispatcher-19] [INFO] Recovery - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 1.851 ms, resetting timer for the next recovery snapshot 08:54:19,105 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:19,105 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:54:19,108 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 08:54:19,108 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 08:54:19,108 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 08:54:19,110 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,110 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,110 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:54:19,110 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:54:19,110 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:54:19,110 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,110 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:54:19,111 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 2.155 ms, resetting timer for the next recovery snapshot 08:54:19,114 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:54:19,114 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:54:19,114 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,114 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,116 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [-3965719788399076101] 08:54:19,116 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 08:54:19,116 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:54:19,116 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,118 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:19,118 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:54:19,119 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:19,119 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:19,120 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:54:19,130 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:19,131 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:19,133 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,142 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1398397746] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,142 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:54:19,142 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:54:19,153 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:54:19,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,153 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,155 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:19,156 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@4782ecbd} 08:54:19,157 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 08:54:19,157 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:19,159 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:19,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:19,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,161 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,162 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:54:19,162 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,163 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:54:19,164 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:54:19,164 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:54:19,165 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:54:19,165 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#591922346 created and ready for shard:member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,165 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:54:19,172 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1616790678 created and ready for shard:member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,172 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1621071423] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,172 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#115186589] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,172 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#591922346] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,172 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1616790678] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,173 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:54:19,173 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#23106831] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,175 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:54:19,175 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:54:19,175 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:54:19,176 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:19,176 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:54:19,177 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,175 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:19,177 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,178 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,177 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,178 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:19,178 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,178 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,178 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,179 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,179 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:19,182 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:19,182 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@200c28fb} 08:54:19,182 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 08:54:19,183 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:19,183 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,184 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 08:54:19,184 AM [cluster-test-shard-dispatcher-31] [INFO] PekkoRecovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 08:54:19,184 AM [cluster-test-shard-dispatcher-31] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 08:54:19,186 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,186 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,186 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#880630056] 08:54:19,186 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,186 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#880630056] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,187 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:54:19,187 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#64534020 created and ready for shard:member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,187 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#64534020] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,187 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 08:54:19,187 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 08:54:19,187 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 08:54:19,188 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:54:19,188 AM [cluster-test-shard-dispatcher-31] [INFO] Recovery - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 3.218 ms, resetting timer for the next recovery snapshot 08:54:19,188 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,188 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,188 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1769088409] 08:54:19,189 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:54:19,189 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1769088409] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,189 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1582698758 created and ready for shard:member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,190 AM [cluster-test-shard-dispatcher-32] [INFO] RaftActor - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:54:19,190 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1582698758] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,191 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:54:19,192 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:54:19,192 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:19,194 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:54:19,194 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:54:19,194 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,194 AM [cluster-test-shard-dispatcher-31] [DEBUG] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,194 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:54:19,194 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,194 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,194 AM [cluster-test-shard-dispatcher-13] [INFO] Recovery - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 7.013 ms, resetting timer for the next recovery snapshot 08:54:19,195 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:19,195 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:54:19,195 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 08:54:19,195 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 08:54:19,195 AM [cluster-test-shard-dispatcher-32] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 08:54:19,195 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:54:19,198 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:54:19,198 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:54:19,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,198 AM [cluster-test-shard-dispatcher-13] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,198 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:54:19,198 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,198 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:54:19,199 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 08:54:19,199 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 08:54:19,199 AM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 08:54:19,199 AM [cluster-test-shard-dispatcher-32] [INFO] Recovery - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 3.733 ms, resetting timer for the next recovery snapshot 08:54:19,202 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,202 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,202 AM [cluster-test-shard-dispatcher-28] [INFO] Recovery - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 3.396 ms, resetting timer for the next recovery snapshot 08:54:19,202 AM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,202 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:54:19,202 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:54:19,203 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:54:19,203 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,203 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:54:19,209 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [2795521493701316861] 08:54:19,209 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 08:54:19,211 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:54:19,211 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:54:19,211 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,211 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,211 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:54:19,211 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,211 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,212 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:54:19,212 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:19,212 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:54:19,212 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:19,212 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:19,224 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:19,226 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:19,230 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,237 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:54:19,237 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:54:19,241 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1030690484] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,245 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:54:19,245 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,246 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,252 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:19,257 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 08:54:19,257 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:19,259 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:19,260 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@1f3a3024} 08:54:19,261 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:19,261 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,263 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,264 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,266 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:54:19,266 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:54:19,266 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:54:19,266 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1889378112] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,266 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-471000627] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,266 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1488973407 created and ready for shard:member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,266 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1488973407] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,267 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:54:19,267 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:54:19,267 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:54:19,267 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1373506896] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,267 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#653930198 created and ready for shard:member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,267 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown-notifier#653930198] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,268 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:54:19,269 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:54:19,270 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,270 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:54:19,270 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,270 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:54:19,270 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:54:19,272 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:19,272 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,272 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,273 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:19,273 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:19,274 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:19,274 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,274 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,274 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,274 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,274 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@1e742058} 08:54:19,274 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} started 08:54:19,274 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:19,274 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:19,274 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,275 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 08:54:19,275 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 08:54:19,275 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 08:54:19,275 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:19,275 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,275 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,275 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-47634881] 08:54:19,275 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,276 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-47634881] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,276 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-68041845] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,276 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:54:19,276 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1165625972 created and ready for shard:member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,277 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#-1165625972] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,277 AM [cluster-test-shard-dispatcher-32] [INFO] RaftActor - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:54:19,277 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:54:19,277 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Shard created, persistent : true 08:54:19,278 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,278 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,278 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown on shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-68041845] 08:54:19,278 AM [cluster-test-shard-dispatcher-11] [INFO] Recovery - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 3.592 ms, resetting timer for the next recovery snapshot 08:54:19,279 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 08:54:19,279 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 08:54:19,279 AM [cluster-test-shard-dispatcher-30] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 08:54:19,279 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:19,279 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1176854562 created and ready for shard:member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,279 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown-notifier#1176854562] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:19,280 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Starting recovery with journal batch size 1 08:54:19,281 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal open: applyTo=0 08:54:19,281 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,281 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,281 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,281 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:54:19,281 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:54:19,281 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:54:19,281 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,282 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:54:19,282 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Recovery completed - Switching actor to Follower - last log index = 0, last log term = 1, snapshot index = -1, snapshot term = -1, journal size = 1 08:54:19,282 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 08:54:19,282 AM [cluster-test-shard-dispatcher-32] [INFO] PekkoRecovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 08:54:19,282 AM [cluster-test-shard-dispatcher-32] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 08:54:19,282 AM [cluster-test-shard-dispatcher-30] [INFO] Recovery - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 3.593 ms, resetting timer for the next recovery snapshot 08:54:19,285 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Local TermInfo store seeded with TermInfo{term=1} 08:54:19,285 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: taking snapshot to clear Pekko persistence to 2 08:54:19,285 AM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Taking snapshot on entry with index -1 08:54:19,286 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:54:19,286 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:54:19,286 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,286 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,286 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:54:19,286 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,286 AM [cluster-test-shard-dispatcher-32] [INFO] Recovery - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 4.164 ms, resetting timer for the next recovery snapshot 08:54:19,286 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,287 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:54:19,288 AM [cluster-test-shard-dispatcher-29] [INFO] Recovery - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Snapshot completed in 3.448 ms, resetting timer for the next recovery snapshot 08:54:19,291 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,291 AM [cluster-test-shard-dispatcher-32] [DEBUG] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,291 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:54:19,291 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:54:19,291 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:54:19,291 AM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,291 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,291 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:54:19,292 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,292 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:54:19,292 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,292 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown is initialized 08:54:19,292 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from null to Follower 08:54:19,292 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,292 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , registered listener pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:19,292 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from null to Follower 08:54:19,461 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 08:54:19,461 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:54:19,461 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,462 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,462 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 08:54:19,462 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 08:54:19,503 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 08:54:19,503 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:54:19,503 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,503 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,504 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 08:54:19,504 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 08:54:19,511 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 08:54:19,511 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:54:19,511 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,511 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,511 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 08:54:19,511 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 08:54:19,522 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 2 08:54:19,522 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower) :- Switching from behavior Follower to Candidate, election term: 2 08:54:19,522 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: false, hasLeader : false 08:54:19,522 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:19,522 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Follower to Candidate 08:54:19,522 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Follower to Candidate 08:54:19,852 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 08:54:19,875 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 08:54:19,882 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 08:54:19,901 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 3 08:54:20,70 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:54:20,70 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:54:20,70 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:20,70 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:20,70 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,70 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,70 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,70 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,70 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,70 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,70 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,70 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:20,71 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:20,71 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-20] [INFO] PeerInfos - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,71 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,72 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,72 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,72 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:20,77 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:20,77 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:20,77 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:20,77 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-31] [INFO] PeerInfos - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,77 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:20,146 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:20,146 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:20,146 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:20,146 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown to pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,146 AM [cluster-test-shard-dispatcher-33] [INFO] PeerInfos - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Peer address for peer member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,190 AM [cluster-test-shard-dispatcher-33] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 08:54:20,192 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:20,194 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:20,203 AM [cluster-test-shard-dispatcher-33] [INFO] Candidate - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:54:20,204 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 08:54:20,204 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:54:20,204 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 08:54:20,204 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 08:54:20,204 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6cd740a6 08:54:20,204 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 08:54:20,206 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:54:20,206 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@70cb283c 08:54:20,206 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:54:20,206 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:54:20,206 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:54:20,206 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@3f7c2bb8 08:54:20,209 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 08:54:20,209 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:54:20,209 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 08:54:20,209 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 08:54:20,221 AM [cluster-test-shard-dispatcher-33] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 08:54:20,222 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:20,222 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:20,227 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:54:20,231 AM [cluster-test-shard-dispatcher-33] [INFO] Candidate - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:54:20,231 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 08:54:20,231 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:54:20,231 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 08:54:20,231 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 08:54:20,231 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6ea5008d 08:54:20,232 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 08:54:20,232 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4f1707fa 08:54:20,233 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:54:20,233 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:54:20,233 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:54:20,233 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:54:20,233 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@34997cf 08:54:20,235 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 08:54:20,235 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:54:20,235 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 08:54:20,235 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 08:54:20,273 AM [cluster-test-shard-dispatcher-33] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 08:54:20,275 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:20,275 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:20,281 AM [cluster-test-shard-dispatcher-33] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): Starting new election term 4 08:54:20,281 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:54:20,281 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 08:54:20,281 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:54:20,281 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 08:54:20,282 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 08:54:20,282 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6210a34c 08:54:20,282 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 08:54:20,283 AM [cluster-test-shard-dispatcher-28] [INFO] Follower - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:20,283 AM [cluster-test-shard-dispatcher-30] [INFO] Follower - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Follower): Term 4 in "RequestVote{term=4, candidateId=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown, lastLogIndex=0, lastLogTerm=1}" message is greater than follower's term 1 - updating term 08:54:20,283 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:54:20,283 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:54:20,284 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@6b751f57 08:54:20,284 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 08:54:20,285 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:54:20,285 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@72c0eeb4 08:54:20,285 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 08:54:20,289 AM [cluster-test-shard-dispatcher-11] [INFO] Candidate - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 08:54:20,289 AM [cluster-test-shard-dispatcher-33] [INFO] RaftActorBehavior - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 08:54:20,289 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:54:20,289 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 08:54:20,289 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:54:20,289 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: became leader with frontend state for [] 08:54:20,289 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from Candidate to PreLeader 08:54:20,289 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 08:54:20,289 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@189fadc3 08:54:20,289 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from Candidate to PreLeader 08:54:20,289 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 08:54:20,289 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testFlipMemberVotingStatesWithVotingMembersDown is ready 08:54:20,291 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:54:20,291 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:54:20,291 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@618d7427 08:54:20,291 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 08:54:20,292 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@564047f0 08:54:20,292 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:54:20,292 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 08:54:20,292 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done false 08:54:20,292 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown status sync done true 08:54:20,293 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActorBehavior - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 08:54:20,293 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: onLeadershipChange, isLeader: true, hasLeader : true 08:54:20,293 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown , received role change from PreLeader to Leader 08:54:20,294 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown from PreLeader to Leader 08:54:20,294 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testFlipMemberVotingStatesWithVotingMembersDown is ready 08:54:20,294 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 08:54:20,294 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:20,294 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} is now ready 08:54:20,294 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1008487736] 08:54:20,294 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:20,294 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-711227713] 08:54:20,294 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-711227713] 08:54:20,295 AM [main] [INFO] ClusterAdminRpcService - Flip member voting states for all shards 08:54:20,295 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,295 AM [main] [DEBUG] ClusterAdminRpcService - Sending message to all shards [cars, people] for data store oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,295 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:54:20,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=cars] 08:54:20,296 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:54:20,296 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:20,296 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: onFlipShardMembersVotingStatus: FlipShardMembersVotingStatus [shardName=people] 08:54:20,296 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:54:20,296 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:54:20,296 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:54:20,297 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,297 AM [cluster-test-pekko.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,297 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,297 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,298 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,298 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard people 08:54:20,298 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Config 08:54:20,298 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 1 08:54:20,298 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,298 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard cars 08:54:20,298 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Operational 08:54:20,298 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 2 08:54:20,298 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,298 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-config-member-1: ChangeServersVotingStatus succeeded for shard cars 08:54:20,298 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=cars}, type Config 08:54:20,298 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 3 08:54:20,300 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown} from local shard pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,300 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: ChangeServersVotingStatus succeeded for shard people 08:54:20,300 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - onSuccess for shard ShardName{value=people}, type Operational 08:54:20,300 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ClusterAdminRpcService - checkIfComplete: expected 4, actual 4 08:54:20,300 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1008487736] 08:54:20,300 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:20,301 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-711227713] 08:54:20,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-711227713] 08:54:20,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1008487736] 08:54:20,301 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : people 08:54:20,301 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1539810658] 08:54:20,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1539810658] 08:54:20,301 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1059607326] 08:54:20,302 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : cars 08:54:20,302 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1450511830] 08:54:20,302 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-1450511830] 08:54:20,302 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#1059607326] 08:54:20,303 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: findLocalShard : people 08:54:20,303 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1682178522] 08:54:20,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1682178522] 08:54:20,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1398397746] 08:54:20,303 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:54:20,303 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1621071423] 08:54:20,303 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1621071423] 08:54:20,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#1398397746] 08:54:20,304 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : people 08:54:20,304 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#115186589] 08:54:20,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#115186589] 08:54:20,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#23106831] 08:54:20,304 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : cars 08:54:20,304 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#880630056] 08:54:20,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#880630056] 08:54:20,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#23106831] 08:54:20,304 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: findLocalShard : people 08:54:20,304 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1769088409] 08:54:20,304 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#1769088409] 08:54:20,305 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1030690484] 08:54:20,305 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:54:20,305 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1373506896] 08:54:20,305 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#1373506896] 08:54:20,305 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1030690484] 08:54:20,305 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : people 08:54:20,305 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1889378112] 08:54:20,305 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown#1889378112] 08:54:20,305 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-471000627] 08:54:20,306 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : cars 08:54:20,306 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-47634881] 08:54:20,306 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown#-47634881] 08:54:20,306 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown#-471000627] 08:54:20,306 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-3: findLocalShard : people 08:54:20,306 AM [cluster-test-pekko.actor.default-dispatcher-21] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-68041845] 08:54:20,306 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-oper_testFlipMemberVotingStatesWithVotingMembersDown/member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown#-68041845] 08:54:20,306 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown#-1008487736] 08:54:20,306 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:20,306 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-711227713] 08:54:20,306 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testFlipMemberVotingStatesWithVotingMembersDown/member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown#-711227713] 08:54:20,307 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:20,307 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,307 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,307 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 08:54:20,307 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:54:20,308 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:54:20,308 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:54:20,308 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:54:20,308 AM [cluster-test-pekko.actor.default-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:54:20,308 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:54:20,309 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:20,309 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,309 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,309 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:54:20,309 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:54:20,309 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:54:20,309 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:54:20,309 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:54:20,309 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:54:20,309 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:54:20,310 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:20,310 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:20,314 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:20,314 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:20,314 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:20,315 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:20,316 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:54:20,329 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:20,329 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,329 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,329 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:54:20,330 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:54:20,330 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:54:20,331 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:54:20,331 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:54:20,331 AM [cluster-test-pekko.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:54:20,331 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:54:20,332 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:20,332 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,333 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:20,333 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:54:20,333 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:54:20,333 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:54:20,333 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:54:20,333 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:54:20,334 AM [cluster-test-pekko.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:54:20,334 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:54:20,334 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:20,335 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:20,908 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorVotingConfigSupport - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 08:54:20,918 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorVotingConfigSupport - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 08:54:20,919 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorVotingConfigSupport - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-5-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-6-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-3-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false, member-4-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=true, member-2-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown=false}, serversVisited=[]] 08:54:20,919 AM [cluster-test-shard-dispatcher-33] [WARN] RaftActorVotingConfigSupport - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Timeout occured while replicating the new server configuration for ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-2-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-3-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=false, member-6-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-4-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true, member-5-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown=true}, serversVisited=[]] 08:54:21,89 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:54:21,329 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [2795521493701316861] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:21,348 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [-3965719788399076101] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:21,352 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:21,352 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:21,354 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:54:21.356] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:21,360 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:21,360 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:21,361 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown 08:54:21,361 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:54:21,361 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2ab7e466 08:54:21,361 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@66c68f30 08:54:21,361 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1174036b 08:54:21,361 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1174036b 08:54:21,361 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 08:54:21,363 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@662f2bd6 08:54:21,363 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@662f2bd6 08:54:21,363 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 08:54:21,828 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 08:54:21,829 AM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 467.7 ms 08:54:21,829 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 08:54:21,829 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:54:21,830 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:54:21,888 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 08:54:21,888 AM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 527.2 ms 08:54:21,888 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 08:54:21,889 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:54:21,889 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-1-shard-people-config_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:54:21,890 AM [cluster-test-pekko.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:54:21,890 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:54:21,890 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testFlipMemberVotingStatesWithVotingMembersDown, generation=0} 08:54:21,891 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:21,891 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown 08:54:21,891 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:54:21,891 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7c8a17eb 08:54:21,891 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c21f768 08:54:21,891 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c21f768 08:54:21,891 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 08:54:21,892 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@232da977 08:54:21,892 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@232da977 08:54:21,892 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: pauseLeader successfully completed - doing transfer 08:54:21,892 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@75a90fc4 08:54:22,209 AM [cluster-test-shard-dispatcher-11] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 08:54:22,209 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 317.6 ms 08:54:22,209 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 08:54:22,210 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:54:22,210 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:54:22,448 AM [cluster-test-shard-dispatcher-33] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: leader transfer aborted 08:54:22,448 AM [cluster-test-shard-dispatcher-33] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Failed to transfer leadership in 556.3 ms 08:54:22,448 AM [cluster-test-shard-dispatcher-33] [DEBUG] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: In unpauseLeader 08:54:22,449 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: Stopping Shard 08:54:22,449 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testFlipMemberVotingStatesWithVotingMembersDown: journal closed 08:54:22,450 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:54:22,450 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:54:22,451 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:22,452 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:22,454 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:54:22.456] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:22,480 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:54:22,488 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [1272376065060401400] 08:54:22,488 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:54:22,490 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:54:22,490 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:54:22,491 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:22,491 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:22,503 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:22,506 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:22,507 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:54:22,507 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:54:22,507 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:54:22,510 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 08:54:22,521 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1089094770] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,521 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:54:22,521 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:54:22,522 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:54:22,523 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,523 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:22,524 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:22,524 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,524 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:22,527 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:54:22,527 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 08:54:22,528 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:22,528 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@6808f00e} 08:54:22,529 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:22,529 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,529 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,529 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,529 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica on shard member-1-shard-cars-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1701773462] 08:54:22,529 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:22,529 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:22,529 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:22,529 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:22,529 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica on shard member-1-shard-people-config_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#-1752507853] 08:54:22,530 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 08:54:22,530 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica#-1752507853] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,530 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#-366768960 created and ready for shard:member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:22,530 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica-notifier#-366768960] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,530 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:54:22,531 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 08:54:22,533 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 08:54:22,534 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 08:54:22,534 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1701773462] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,534 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#58400779 created and ready for shard:member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,534 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica-notifier#58400779] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,534 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:54:22,535 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 08:54:22,537 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-1587282152] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,537 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:54:22,537 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:54:22,537 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:22,541 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:54:22,541 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:54:22,541 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testRemoveShardLeaderReplica is initialized 08:54:22,541 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,541 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,542 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 08:54:22,542 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 08:54:22,542 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from null to Follower 08:54:22,542 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 08:54:22,543 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 08:54:22,548 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:22,550 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:54:22,551 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,552 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,553 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:22,553 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,553 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,553 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@6b5603b0} 08:54:22,553 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:22,553 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,554 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,554 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,555 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-1-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#-731135288] 08:54:22,555 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,555 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,555 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,555 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-1-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#1340951377] 08:54:22,555 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:54:22,556 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 08:54:22,556 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica#-731135288] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,556 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:54:22,556 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testRemoveShardLeaderReplica is initialized 08:54:22,556 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-53426920 created and ready for shard:member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,557 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-53426920] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,557 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica#1340951377] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,557 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 08:54:22,557 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:54:22,557 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:54:22,558 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 08:54:22,565 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 08:54:22,565 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#-1056598555 created and ready for shard:member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,565 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica-notifier#-1056598555] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,566 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:22,566 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:22,568 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:54:22,569 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:54:22,569 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,569 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,569 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 08:54:22,570 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:54:22,570 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testRemoveShardLeaderReplica is initialized 08:54:22,570 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:54:22,570 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testRemoveShardLeaderReplica is initialized 08:54:22,575 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,575 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,575 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 08:54:22,575 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 08:54:22,575 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 08:54:22,575 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 08:54:22,576 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:54:22,582 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,582 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,582 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 08:54:22,582 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 08:54:22,603 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 08:54:22,603 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 08:54:22,628 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [4144016623312962373] 08:54:22,629 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 08:54:22,634 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:22,634 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:54:22,635 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:22,635 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:22,647 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:22,649 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:22,660 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 08:54:22,670 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#45492964] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,670 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:54:22,671 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:54:22,672 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:54:22,672 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,672 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testRemoveShardLeaderReplica 08:54:22,674 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:54:22,676 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@373f5625} 08:54:22,676 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 08:54:22,677 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:22,679 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:22,680 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:22,680 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,683 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 08:54:22,685 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testRemoveShardLeaderReplica 08:54:22,686 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-273804386] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,686 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica#-1863315556] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,686 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 08:54:22,686 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 08:54:22,686 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-374700917 created and ready for shard:member-2-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,686 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#2046455738 created and ready for shard:member-2-shard-people-config_testRemoveShardLeaderReplica 08:54:22,686 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActor - member-2-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:54:22,686 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-539056929] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,687 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica-notifier#-374700917] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,687 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica-notifier#2046455738] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,687 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:54:22,689 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 08:54:22,689 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:54:22,690 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:54:22,691 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:22,692 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:54:22,693 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 08:54:22,694 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:54:22,694 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,694 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:22,695 AM [cluster-test-shard-dispatcher-27] [INFO] PekkoRecovery - member-2-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:54:22,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,696 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 08:54:22,696 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:54:22,696 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:22,696 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,696 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,697 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:22,697 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:54:22,698 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:54:22,699 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,699 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 08:54:22,699 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,699 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 08:54:22,699 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:22,699 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:54:22,699 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testRemoveShardLeaderReplica is initialized 08:54:22,700 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 08:54:22,700 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testRemoveShardLeaderReplica from null to Follower 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@6df8e66f} 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-2-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica#873359528] 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,700 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-2-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica#1314244500] 08:54:22,701 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica#873359528] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,701 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica#1314244500] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,701 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 08:54:22,701 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#934049066 created and ready for shard:member-2-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,701 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica-notifier#934049066] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,702 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActor - member-2-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:54:22,702 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 08:54:22,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:54:22,704 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testRemoveShardLeaderReplica is initialized 08:54:22,704 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,704 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,704 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 08:54:22,704 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,704 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 08:54:22,705 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 08:54:22,705 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 08:54:22,705 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-1393480748 created and ready for shard:member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,706 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica-notifier#-1393480748] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,706 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:54:22,707 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 08:54:22,707 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:22,708 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:22,709 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:54:22,709 AM [cluster-test-shard-dispatcher-30] [INFO] PekkoRecovery - member-2-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:54:22,710 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:54:22,710 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testRemoveShardLeaderReplica is initialized 08:54:22,710 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,710 AM [cluster-test-shard-dispatcher-30] [DEBUG] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,710 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 08:54:22,710 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,710 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 08:54:22,710 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:54:22,710 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,710 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,710 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 08:54:22,710 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,711 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 08:54:22,711 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:54:22,711 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testRemoveShardLeaderReplica is initialized 08:54:22,711 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 08:54:22,711 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 08:54:22,722 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [1755635857898391275] 08:54:22,722 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 08:54:22,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:22,728 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:54:22,729 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:22,729 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:22,740 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:22,741 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:22,743 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testRemoveShardLeaderReplica 08:54:22,751 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:54:22,751 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:54:22,752 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1494004912] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,753 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:54:22,753 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,754 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testRemoveShardLeaderReplica 08:54:22,764 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:54:22,765 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:54:22,765 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} started 08:54:22,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@70d98296} 08:54:22,766 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:22,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:22,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,766 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testRemoveShardLeaderReplica 08:54:22,767 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Shard created, persistent : true 08:54:22,767 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-377975618] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,767 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:54:22,767 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1171560981 created and ready for shard:member-3-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,767 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-3-shard-cars-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:54:22,767 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica-notifier#-1171560981] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,769 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica#1791214482] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,769 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Shard created, persistent : true 08:54:22,769 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardLeaderReplica: journal open: applyTo=0 08:54:22,769 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:22,769 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#359201048 created and ready for shard:member-3-shard-people-config_testRemoveShardLeaderReplica 08:54:22,769 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica-notifier#359201048] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,769 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-3-shard-people-config_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:54:22,771 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardLeaderReplica: journal open: applyTo=0 08:54:22,772 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:22,772 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:22,773 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:22,773 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,773 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,773 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:22,773 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:54:22,776 AM [cluster-test-shard-dispatcher-11] [INFO] PekkoRecovery - member-3-shard-cars-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:54:22,776 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-3-shard-people-config_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:54:22,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,777 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,777 AM [cluster-test-shard-dispatcher-11] [DEBUG] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,777 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:22,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:54:22,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testRemoveShardLeaderReplica is initialized 08:54:22,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:54:22,777 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testRemoveShardLeaderReplica is initialized 08:54:22,777 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , received role change from null to Follower 08:54:22,778 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 08:54:22,778 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testRemoveShardLeaderReplica from null to Follower 08:54:22,778 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , received role change from null to Follower 08:54:22,778 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:22,778 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica 08:54:22,778 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testRemoveShardLeaderReplica from null to Follower 08:54:22,779 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testRemoveShardLeaderReplica 08:54:22,780 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica#-1382949256] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,781 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:54:22,781 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:54:22,782 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:54:22,782 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,783 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,783 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:22,783 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,783 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,784 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testRemoveShardLeaderReplica: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 08:54:22,785 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@70d386ba} 08:54:22,785 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} started 08:54:22,785 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:54:22,786 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:22,786 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,787 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#887741977] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,787 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 08:54:22,788 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1818816803 created and ready for shard:member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,788 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActor - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:54:22,788 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica-notifier#1818816803] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,789 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 08:54:22,790 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,790 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,790 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-cars-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica on shard member-3-shard-cars-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica#887741977] 08:54:22,790 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,791 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:22,791 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Shard created, persistent : true 08:54:22,791 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#-482994126] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,792 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#-860724028 created and ready for shard:member-3-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,792 AM [cluster-test-shard-dispatcher-15] [INFO] RaftActor - member-3-shard-people-oper_testRemoveShardLeaderReplica: Starting recovery with journal batch size 1 08:54:22,792 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,792 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,792 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: updated peer member-1-shard-people-oper_testRemoveShardLeaderReplica on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica on shard member-3-shard-people-oper_testRemoveShardLeaderReplica whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica#-482994126] 08:54:22,792 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica-notifier#-860724028] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:22,793 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardLeaderReplica: journal open: applyTo=0 08:54:22,793 AM [cluster-test-shard-dispatcher-29] [INFO] PekkoRecovery - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:54:22,794 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:54:22,794 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testRemoveShardLeaderReplica is initialized 08:54:22,794 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,794 AM [cluster-test-shard-dispatcher-29] [DEBUG] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,794 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , received role change from null to Follower 08:54:22,794 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardLeaderReplica: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:22,794 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:22,794 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 08:54:22,794 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testRemoveShardLeaderReplica from null to Follower 08:54:22,800 AM [cluster-test-shard-dispatcher-15] [INFO] PekkoRecovery - member-3-shard-people-oper_testRemoveShardLeaderReplica: Local TermInfo store seeded with TermInfo{term=0} 08:54:22,800 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:54:22,800 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testRemoveShardLeaderReplica is initialized 08:54:22,800 AM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,800 AM [cluster-test-shard-dispatcher-15] [DEBUG] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,800 AM [cluster-test-shard-dispatcher-15] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-1-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:22,801 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , received role change from null to Follower 08:54:22,801 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testRemoveShardLeaderReplica , registered listener pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica 08:54:22,801 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testRemoveShardLeaderReplica from null to Follower 08:54:22,892 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 08:54:22,892 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:22,892 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,892 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,892 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 08:54:22,892 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Follower to Candidate 08:54:22,901 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 08:54:22,901 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:22,901 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,901 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,901 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 08:54:22,901 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 08:54:22,911 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 08:54:22,911 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:22,911 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:22,911 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:22,911 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 08:54:22,911 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Follower to Candidate 08:54:23,13 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 1 08:54:23,13 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:23,13 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:23,13 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:23,13 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Follower to Candidate 08:54:23,14 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Follower to Candidate 08:54:23,281 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 08:54:23,291 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 08:54:23,311 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 08:54:23,422 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 2 08:54:23,509 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:54:23,510 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:54:23,510 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:23,511 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:23,511 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:23,511 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-22] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:54:23,511 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,512 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,512 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:23,512 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,512 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,512 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,512 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,512 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,512 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,512 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-1-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:23,518 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:54:23,518 AM [cluster-test-shard-dispatcher-29] [INFO] PeerInfos - member-2-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-3-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:23,662 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:23,662 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:23,662 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:23,662 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,662 AM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:54:23,663 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-people-config_testRemoveShardLeaderReplica 08:54:23,663 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,663 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-people-config_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-config_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-people-config_testRemoveShardLeaderReplica 08:54:23,663 AM [cluster-test-shard-dispatcher-11] [INFO] PeerInfos - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-cars-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,663 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,663 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testRemoveShardLeaderReplica with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica to pekko://cluster-test/user/shardmanager-oper_testRemoveShardLeaderReplica/member-3-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,663 AM [cluster-test-shard-dispatcher-30] [INFO] PeerInfos - member-3-shard-people-oper_testRemoveShardLeaderReplica: Peer address for peer member-2-shard-people-oper_testRemoveShardLeaderReplica set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testRemoveShardLeaderReplica/member-2-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,693 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 08:54:23,695 AM [cluster-test-shard-dispatcher-31] [INFO] Follower - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:23,695 AM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:23,702 AM [cluster-test-shard-dispatcher-24] [INFO] Candidate - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 08:54:23,704 AM [cluster-test-shard-dispatcher-15] [INFO] Follower - member-3-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:23,704 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-people-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:23,706 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 08:54:23,707 AM [cluster-test-shard-dispatcher-21] [INFO] Follower - member-2-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:23,708 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:54:23,708 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:23,708 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 08:54:23,708 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-people-config_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:23,708 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 08:54:23,709 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@a2fe5b1 08:54:23,709 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done false 08:54:23,709 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@79549275 08:54:23,709 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 08:54:23,710 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@ada39bb 08:54:23,710 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 08:54:23,745 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:54:23,746 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:54:23,746 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:23,746 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4e80a84 08:54:23,747 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:54:23,748 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:23,748 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 08:54:23,748 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 08:54:23,748 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testRemoveShardLeaderReplica status sync done false 08:54:23,748 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@d78ec1 08:54:23,748 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Candidate to Leader 08:54:23,748 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 08:54:23,748 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} is now ready 08:54:23,748 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1089094770] 08:54:23,748 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@53ac4e23 08:54:23,748 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done false 08:54:23,749 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@20f0007d 08:54:23,749 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:23,749 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1701773462] 08:54:23,749 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1701773462] 08:54:23,749 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testRemoveShardLeaderReplica status sync done false 08:54:23,749 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@56da62eb 08:54:23,749 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 08:54:23,749 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@508a00ce 08:54:23,750 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 08:54:23,750 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testRemoveShardLeaderReplica status sync done false 08:54:23,750 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#45492964] 08:54:23,750 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:54:23,750 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-273804386] 08:54:23,750 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 08:54:23,750 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 08:54:23,750 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Candidate to Leader 08:54:23,750 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-273804386] 08:54:23,750 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1494004912] 08:54:23,750 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:54:23,751 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-377975618] 08:54:23,751 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-377975618] 08:54:23,751 AM [main] [INFO] ClusterAdminRpcService - Removing replica for shard ShardName{value=cars} memberName MemberName{value=member-1}, datastoreType Config 08:54:23,751 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: onRemoveShardReplica: RemoveShardReplica [shardName=cars, memberName=MemberName{name=member-1}] 08:54:23,752 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 08:54:23,752 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Found primary for cars: LocalPrimaryShardFound [primaryPath=pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1701773462, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4b6a41c8] 08:54:23,753 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-1: Sending RemoveServer message to peer pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1701773462 for shard member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,753 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Received RemoveServerReply{status=OK, leaderHint=member-1-shard-cars-config_testRemoveShardLeaderReplica} for shard cars 08:54:23,753 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Leader shard successfully removed the replica shard cars 08:54:23,754 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] ClusterAdminRpcService - Successfully removed replica for shard ShardName{value=cars} 08:54:23,754 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#45492964] 08:54:23,754 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:54:23,754 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-273804386] 08:54:23,754 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-273804386] 08:54:23,758 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true], ServerInfo[peerId=member-3-shard-cars-config_testRemoveShardLeaderReplica, isVoting=true]]} has been successfully replicated to a majority of followers 08:54:23,759 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1 : Sending Shutdown to Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1701773462] with 10000 ms timeout 08:54:23,759 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1 : Local Shard replica for shard cars has been removed 08:54:23,759 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: persisting the shard list [people] 08:54:23,759 AM [cluster-test-shard-dispatcher-24] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@45ada6d5 08:54:23,759 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@45ada6d5 08:54:23,759 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 08:54:23,761 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testRemoveShardLeaderReplica status sync done true 08:54:23,761 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@1d290d9b 08:54:23,761 AM [cluster-test-shard-dispatcher-24] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader transfer complete - waiting for new leader 08:54:23,761 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2145cb5d 08:54:23,762 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,762 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 08:54:23,762 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@76eee8ca 08:54:23,765 AM [cluster-test-shard-dispatcher-27] [INFO] Candidate - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate): Starting new election term 4 08:54:23,765 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Follower) :- Switching from behavior Follower to Candidate, election term: 4 08:54:23,765 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:23,765 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:23,765 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Follower to Candidate 08:54:23,765 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7ff025af 08:54:23,765 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Follower to Candidate 08:54:23,766 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-cars-config_testRemoveShardLeaderReplica (Follower): Term 4 in "RequestVote{term=4, candidateId=member-2-shard-cars-config_testRemoveShardLeaderReplica, lastLogIndex=0, lastLogTerm=3}" message is greater than follower's term 3 - updating term 08:54:23,771 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 08:54:23,771 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:23,772 AM [cluster-test-shard-dispatcher-27] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: became leader with frontend state for [] 08:54:23,772 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testRemoveShardLeaderReplica , received role change from Candidate to Leader 08:54:23,772 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@5bd79185 08:54:23,772 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testRemoveShardLeaderReplica from Candidate to Leader 08:54:23,772 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 08:54:23,772 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@49d30fe 08:54:23,772 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testRemoveShardLeaderReplica is ready 08:54:23,772 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done false 08:54:23,774 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testRemoveShardLeaderReplica status sync done true 08:54:23,801 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate): Starting new election term 3 08:54:23,803 AM [cluster-test-shard-dispatcher-27] [INFO] Follower - member-2-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:23,803 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-3-shard-cars-oper_testRemoveShardLeaderReplica (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testRemoveShardLeaderReplica, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:23,808 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-273804386] 08:54:23,808 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#45492964] 08:54:23,808 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:54:23,808 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-273804386] 08:54:23,809 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-2-shard-cars-config_testRemoveShardLeaderReplica#-273804386] 08:54:23,809 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#1494004912] 08:54:23,809 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:54:23,809 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-377975618] 08:54:23,809 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-3-shard-cars-config_testRemoveShardLeaderReplica#-377975618] 08:54:23,810 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica#-1089094770] 08:54:23,810 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:23,810 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Local shard cars not found - shards present: [people] 08:54:23,811 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:54:23,811 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:54:23,811 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:23,811 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: became leader with frontend state for [] 08:54:23,811 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Candidate to Leader 08:54:23,811 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,811 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testRemoveShardLeaderReplica 08:54:23,811 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@492ee08f 08:54:23,811 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 08:54:23,811 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Candidate to Leader 08:54:23,811 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 08:54:23,812 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 08:54:23,812 AM [cluster-test-shard-dispatcher-15] [INFO] Shard - member-3-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 08:54:23,812 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-people-config_testRemoveShardLeaderReplica: journal closed 08:54:23,813 AM [cluster-test-shard-dispatcher-15] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testRemoveShardLeaderReplica: journal closed 08:54:23,813 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:54:23,813 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:54:23,813 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@19a97a23 08:54:23,814 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 08:54:23,814 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 08:54:23,814 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 08:54:23,814 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testRemoveShardLeaderReplica 08:54:23,814 AM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testRemoveShardLeaderReplica 08:54:23,814 AM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:54:23,814 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-3-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 08:54:23,815 AM [cluster-test-shard-dispatcher-31] [INFO] Shard - member-3-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 08:54:23,815 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 08:54:23,815 AM [cluster-test-shard-dispatcher-31] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testRemoveShardLeaderReplica: journal closed 08:54:23,815 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:54:23,815 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:54:23,816 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:23,816 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done false 08:54:23,816 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:23,816 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2a9077f5 08:54:23,816 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 08:54:23,820 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:23,820 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:23,821 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:23,821 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:23,822 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:54:23,835 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:54:23,835 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testRemoveShardLeaderReplica 08:54:23,835 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testRemoveShardLeaderReplica 08:54:23,836 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:54:23,836 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7f80e2e9 08:54:23,836 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1ca4279d 08:54:23,836 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-config_testRemoveShardLeaderReplica: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1ca4279d 08:54:23,836 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: pauseLeader successfully completed - doing transfer 08:54:23,836 AM [cluster-test-shard-dispatcher-29] [INFO] Shard - member-2-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 08:54:23,837 AM [cluster-test-shard-dispatcher-29] [INFO] EnabledRaftStorage - member-2-shard-people-config_testRemoveShardLeaderReplica: journal closed 08:54:24,61 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testRemoveShardLeaderReplica status sync done true 08:54:24,62 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 311, lastApplied : -1, commitIndex : -1 08:54:24,129 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testRemoveShardLeaderReplica status sync done true 08:54:24,130 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 313, lastApplied : -1, commitIndex : -1 08:54:24,340 AM [cluster-test-shard-dispatcher-29] [INFO] RaftActorBehavior - member-2-shard-cars-config_testRemoveShardLeaderReplica (Leader): Cannot append entries because sender's term 3 is less than 4 08:54:24,341 AM [cluster-test-shard-dispatcher-12] [INFO] AbstractLeader - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader): Term 4 in "AppendEntriesReply{term=4, success=false, followerId=member-2-shard-cars-config_testRemoveShardLeaderReplica, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}" message is greater than leader's term 3 - switching to Follower 08:54:24,344 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to Follower, election term: 4 08:54:24,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: false, hasLeader : false 08:54:24,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:24,344 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testRemoveShardLeaderReplica , received role change from Leader to Follower 08:54:24,344 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@153eff66 08:54:24,344 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - No shard found with member Id member-1-shard-cars-config_testRemoveShardLeaderReplica 08:54:24,344 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testRemoveShardLeaderReplica from Leader to Follower 08:54:24,381 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:24,451 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:24,529 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:54:24,705 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 323, lastApplied : -1, commitIndex : -1 08:54:24,771 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:24,829 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [1755635857898391275] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:24,836 AM [cluster-test-pekko.actor.default-dispatcher-26] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [1755635857898391275] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:25,22 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 317, lastApplied : -1, commitIndex : -1 08:54:25,92 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:25,342 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:25,411 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:54:25,661 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:25,731 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:25,778 AM [cluster-test-shard-dispatcher-12] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: leader not elected in time 08:54:25,778 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testRemoveShardLeaderReplica: Successfully transferred leadership to null in 2.019 s 08:54:25,779 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 08:54:25,779 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testRemoveShardLeaderReplica: journal closed 08:54:25,780 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1 : Successfully shut down Shard actor Actor[pekko://cluster-test/user/shardmanager-config_testRemoveShardLeaderReplica/member-1-shard-cars-config_testRemoveShardLeaderReplica#1701773462] 08:54:25,982 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:26,51 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:26,302 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:26,371 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:26,622 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:26,692 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:54:26,941 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:27,12 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:27,261 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:27,331 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:27,581 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:27,650 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:27,901 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:27,970 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:28,222 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:28,292 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:54:28,542 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:28,611 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:28,858 AM [cluster-test-shard-dispatcher-24] [WARN] Leader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:54:28,859 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:54:28,859 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:28,859 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 08:54:28,859 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:54:28,859 AM [cluster-test-shard-dispatcher-12] [WARN] Leader - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:54:28,859 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-config_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:54:28,859 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:28,859 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 08:54:28,859 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:54:28,861 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:28,861 AM [cluster-test-shard-dispatcher-12] [INFO] IsolatedLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica: IsolatedLeader member-1-shard-people-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 08:54:28,861 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 08:54:28,861 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:28,861 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 08:54:28,861 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 08:54:28,861 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 08:54:28,929 AM [cluster-test-shard-dispatcher-12] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:54:28,929 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:54:28,929 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:28,929 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 08:54:28,930 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:54:28,931 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:28,931 AM [cluster-test-shard-dispatcher-12] [INFO] IsolatedLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica: IsolatedLeader member-1-shard-cars-oper_testRemoveShardLeaderReplica switching from IsolatedLeader to Leader 08:54:28,932 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 3 08:54:28,932 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:28,932 AM [cluster-test-pekko.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from IsolatedLeader to Leader 08:54:28,932 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from IsolatedLeader to Leader 08:54:28,932 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testRemoveShardLeaderReplica is ready 08:54:29,181 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:54:29,252 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 08:54:29,502 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:29,505 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:29,570 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:54:29,630 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 08:54:29,630 AM [cluster-test-pekko.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:54:29,630 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:29,630 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:29,644 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:29,767 AM [cluster-test-pekko.actor.default-dispatcher-26] [WARN] Cluster - pekkoUnreachable Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(pekko://cluster-test@127.0.0.1:2557, Up)]. 08:54:29,767 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:29,767 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:29,821 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:54:29,825 AM [cluster-test-pekko.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:29,891 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:30,141 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:30,218 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 326, lastApplied : -1, commitIndex : -1 08:54:30,462 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:30,531 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 312, lastApplied : -1, commitIndex : -1 08:54:30,780 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:54:30,851 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:31,102 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 321, lastApplied : -1, commitIndex : -1 08:54:31,170 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:31,421 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:31,451 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:31,491 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:31,684 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:31,745 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 323, lastApplied : -1, commitIndex : -1 08:54:31,746 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:31,810 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 318, lastApplied : -1, commitIndex : -1 08:54:32,61 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 316, lastApplied : -1, commitIndex : -1 08:54:32,130 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:32,381 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:32,384 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:32,451 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 320, lastApplied : -1, commitIndex : -1 08:54:32,701 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:32,704 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:32,771 AM [cluster-test-shard-dispatcher-24] [WARN] AbstractLeader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-cars-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:33,21 AM [cluster-test-shard-dispatcher-12] [WARN] AbstractLeader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply{term=3, success=true, followerId=member-2-shard-people-oper_testRemoveShardLeaderReplica, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5}, timeSinceLastActivity : 319, lastApplied : -1, commitIndex : -1 08:54:33,24 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:33,46 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: leader transfer aborted 08:54:33,46 AM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-config_testRemoveShardLeaderReplica: Failed to transfer leadership in 9.210 s 08:54:33,46 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: In unpauseLeader 08:54:33,47 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-config_testRemoveShardLeaderReplica: Stopping Shard 08:54:33,47 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testRemoveShardLeaderReplica: journal closed 08:54:33,48 AM [cluster-test-pekko.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:54:33,48 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:54:33,48 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 08:54:33,49 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testRemoveShardLeaderReplica 08:54:33,49 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testRemoveShardLeaderReplica 08:54:33,49 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:54:33,49 AM [cluster-test-shard-dispatcher-30] [INFO] Shard - member-2-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 08:54:33,49 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-2-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 08:54:33,50 AM [cluster-test-shard-dispatcher-30] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testRemoveShardLeaderReplica: journal closed 08:54:33,50 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 08:54:33,50 AM [cluster-test-pekko.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:54:33,50 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:54:33,51 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:33,51 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:33,345 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:33,724 AM [cluster-test-pekko.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:33,979 AM [cluster-test-shard-dispatcher-24] [WARN] Leader - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:54:33,979 AM [cluster-test-shard-dispatcher-24] [INFO] RaftActorBehavior - member-1-shard-people-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:54:33,979 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:33,979 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 08:54:33,979 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:54:33,985 AM [cluster-test-pekko.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2557,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 08:54:34,49 AM [cluster-test-shard-dispatcher-12] [WARN] Leader - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader): At least 1 followers need to be active, Switching member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:54:34,49 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testRemoveShardLeaderReplica (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 3 08:54:34,49 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testRemoveShardLeaderReplica: onLeadershipChange, isLeader: true, hasLeader : true 08:54:34,49 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testRemoveShardLeaderReplica , received role change from Leader to IsolatedLeader 08:54:34,49 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testRemoveShardLeaderReplica from Leader to IsolatedLeader 08:54:34,69 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:34,69 AM [cluster-test-pekko.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:34,69 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [4144016623312962373] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:34,71 AM [cluster-test-pekko.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:54:34.073] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:34,77 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testRemoveShardLeaderReplica, generation=0} 08:54:34,78 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testRemoveShardLeaderReplica 08:54:34,78 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 1 shards 08:54:34,78 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-people-config_testRemoveShardLeaderReplica: Stopping Shard 08:54:34,79 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-1-shard-people-config_testRemoveShardLeaderReplica: journal closed 08:54:34,80 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:54:34,80 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:54:34,80 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testRemoveShardLeaderReplica, generation=0} 08:54:34,80 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testRemoveShardLeaderReplica 08:54:34,80 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testRemoveShardLeaderReplica 08:54:34,80 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:54:34,81 AM [cluster-test-shard-dispatcher-24] [INFO] Shard - member-1-shard-cars-oper_testRemoveShardLeaderReplica: Stopping Shard 08:54:34,81 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-oper_testRemoveShardLeaderReplica: Stopping Shard 08:54:34,81 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testRemoveShardLeaderReplica: journal closed 08:54:34,81 AM [cluster-test-shard-dispatcher-24] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testRemoveShardLeaderReplica: journal closed 08:54:34,81 AM [cluster-test-pekko.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:54:34,81 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:54:34,82 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:34,82 AM [cluster-test-pekko.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [04/10/2026 08:54:34.086] [cluster-test-pekko.actor.internal-dispatcher-2] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:34,115 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:54:34,123 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [-517157499275219899] 08:54:34,123 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:54:34,125 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:54:34,125 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:54:34,125 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:34,125 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:34,133 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,134 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,136 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,140 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:54:34,140 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:54:34,141 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:54:34,141 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:54:34,141 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:54:34,141 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#1845608651] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,142 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:54:34,142 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,143 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,143 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:34,143 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,143 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,146 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 08:54:34,146 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@1a109115} 08:54:34,147 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 08:54:34,147 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:34,147 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,147 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,148 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,148 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,148 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-638578659] 08:54:34,148 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,148 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,148 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,148 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-1952434418] 08:54:34,148 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 08:54:34,149 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 08:54:34,149 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-638578659] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,149 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,149 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#1163821360 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,149 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#1163821360] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,150 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 08:54:34,150 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 08:54:34,150 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard#-1952434418] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,151 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 08:54:34,151 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#1438688831 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,151 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 08:54:34,151 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,152 AM [cluster-test-pekko.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard-notifier#1438688831] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,153 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard#-959214013] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,153 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:54:34,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:54:34,154 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,154 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,155 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:54:34,155 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,156 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,156 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:34,156 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,156 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,158 AM [cluster-test-shard-dispatcher-13] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,158 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 08:54:34,158 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,159 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,159 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@2623a945} 08:54:34,159 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 08:54:34,159 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,159 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:54:34,159 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 08:54:34,159 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,159 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,159 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 08:54:34,159 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:54:34,159 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard is initialized 08:54:34,160 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} started 08:54:34,160 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 08:54:34,160 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:34,160 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,161 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,161 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,161 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-1838814778] 08:54:34,161 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,161 AM [cluster-test-shard-dispatcher-13] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 08:54:34,161 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,161 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 08:54:34,161 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:54:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 08:54:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 08:54:34,161 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 08:54:34,161 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,161 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@181c2e2d 08:54:34,161 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 08:54:34,162 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 08:54:34,162 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#-1326261112] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,162 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard#-1838814778] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,162 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Shard created, persistent : true 08:54:34,162 AM [cluster-test-shard-dispatcher-13] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 08:54:34,162 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1683273966 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,162 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#-1683273966] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,163 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#1353751310 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,163 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 08:54:34,163 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 08:54:34,163 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard-notifier#1353751310] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,163 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:34,163 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,163 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,163 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 08:54:34,164 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:54:34,164 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 08:54:34,164 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 08:54:34,164 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 08:54:34,164 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,164 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,164 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,164 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7c5d5db3 08:54:34,164 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 08:54:34,164 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatesForSingleNodeShard is ready 08:54:34,164 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} is now ready 08:54:34,164 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Starting recovery with journal batch size 1 08:54:34,165 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard ShardName{value=cars}: {member-1=false} 08:54:34,166 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: journal open: applyTo=0 08:54:34,166 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-1=false}] 08:54:34,166 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard on shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard#-1326261112] 08:54:34,167 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:34,167 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,167 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Received ServerChangeReply{status=INVALID_REQUEST, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,167 AM [cluster-test-pekko.actor.default-dispatcher-11] [ERROR] ClusterAdminRpcService - Failed to change member voting states for shard ShardName{value=cars} java.lang.IllegalArgumentException: The requested voting state change for shard cars is invalid. At least one member must be voting at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$10.onComplete(ShardManager.java:1570) at org.apache.pekko.dispatch.OnComplete.internal(Future.scala:310) at org.apache.pekko.dispatch.OnComplete.internal(Future.scala:307) at org.apache.pekko.dispatch.japi$CallbackBridge.apply(Future.scala:234) at org.apache.pekko.dispatch.japi$CallbackBridge.apply(Future.scala:231) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:517) at org.apache.pekko.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:73) at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:110) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:110) at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59) at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:57) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188) 08:54:34,168 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard#1845608651] 08:54:34,168 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:34,168 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-638578659] 08:54:34,168 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatesForSingleNodeShard/member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard#-638578659] 08:54:34,168 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 08:54:34,169 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,169 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,169 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:54:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@2b67bc8c 08:54:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@2b67bc8c 08:54:34,169 AM [cluster-test-shard-dispatcher-13] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 08:54:34,170 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,170 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,170 AM [cluster-test-shard-dispatcher-13] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatesForSingleNodeShard: journal closed 08:54:34,170 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@717e957e 08:54:34,170 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@717e957e 08:54:34,170 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 08:54:34,171 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatesForSingleNodeShard: journal closed 08:54:34,172 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,173 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,173 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,173 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:54:34,173 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 08:54:34,173 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 08:54:34,174 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:54:34,174 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:54:34,174 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatesForSingleNodeShard, generation=0} 08:54:34,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,174 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:54:34,174 AM [cluster-test-shard-dispatcher-18] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,175 AM [cluster-test-shard-dispatcher-12] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 08:54:34,175 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:34,175 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,175 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,175 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:54:34,175 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 08:54:34,175 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 08:54:34,175 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:54:34,175 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard is initialized 08:54:34,175 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,175 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 08:54:34,175 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,175 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from null to Follower 08:54:34,177 AM [cluster-test-shard-dispatcher-18] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate): Starting new election term 1 08:54:34,177 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:34,177 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,177 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,177 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Follower to Candidate 08:54:34,177 AM [cluster-test-shard-dispatcher-18] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 08:54:34,177 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: onLeadershipChange, isLeader: true, hasLeader : true 08:54:34,177 AM [cluster-test-shard-dispatcher-18] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: became leader with frontend state for [] 08:54:34,177 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , received role change from Candidate to Leader 08:54:34,178 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatesForSingleNodeShard 08:54:34,178 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@7fb96521 08:54:34,178 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard from Candidate to Leader 08:54:34,179 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatesForSingleNodeShard: journal closed 08:54:34,185 AM [cluster-test-shard-dispatcher-18] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: Stopping Shard 08:54:34,185 AM [cluster-test-shard-dispatcher-18] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatesForSingleNodeShard: journal closed 08:54:34,185 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:54:34,185 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:54:34,186 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:34,186 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:34,188 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:54:34,217 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:54:34,227 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2558] with UID [5252155386074546147] 08:54:34,227 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Starting up, Pekko version [1.0.3] ... 08:54:34,229 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [pekko:type=Cluster] 08:54:34,230 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Started up successfully 08:54:34,230 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:34,230 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:34,244 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,244 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 08:54:34,244 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 08:54:34,245 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2558] to [Up] 08:54:34,246 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,250 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 08:54:34,254 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-1 08:54:34,254 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: snapshot not present 08:54:34,255 AM [cluster-test-pekko.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-631104172] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,256 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:54:34,257 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Recovery complete 08:54:34,257 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,258 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Creating local shard: member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,258 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 08:54:34,258 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:34,258 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,258 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,258 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,258 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@3b4e2fcb} 08:54:34,258 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:34,258 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,259 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,259 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,259 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#183280174] 08:54:34,259 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,259 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#183280174] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,259 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,259 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:54:34,261 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 08:54:34,261 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:54:34,262 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:54:34,263 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:54:34,263 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:54:34,264 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:54:34,265 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#709762675 created and ready for shard:member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,265 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-314074738 created and ready for shard:member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,265 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#-254516372] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,265 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#709762675] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,265 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-314074738] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,265 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#1446992631] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,266 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-1 08:54:34,266 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: snapshot not present 08:54:34,268 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,269 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: updated peer member-1-shard-people-config_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard on shard member-1-shard-people-config_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard#-254516372] 08:54:34,270 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Recovery complete 08:54:34,270 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,270 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: Creating local shard: member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,271 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,271 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:34,271 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,271 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,271 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,273 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RecoveringClientActorBehavior - member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 08:54:34,273 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-1: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@4ee009e9} 08:54:34,273 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 08:54:34,274 AM [cluster-test-shard-dispatcher-20] [INFO] PekkoRecovery - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,274 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [cars] 08:54:34,274 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 08:54:34,274 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:34,274 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,275 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,275 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,275 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-1681719228] 08:54:34,275 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,275 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,275 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,275 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-1: updated peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard on member MemberName{name=member-1} with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard on shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard whose actor address is Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#63232640] 08:54:34,275 AM [cluster-test-shard-dispatcher-11] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:54:34,276 AM [cluster-test-shard-dispatcher-11] [INFO] RaftActor - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:54:34,276 AM [cluster-test-shard-dispatcher-21] [INFO] PekkoRecovery - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,277 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard#-1681719228] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,277 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#891446393 created and ready for shard:member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,277 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:54:34,277 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#891446393] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,277 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard#63232640] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,277 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1771775246 created and ready for shard:member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,277 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActor - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:54:34,277 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#1771775246] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,278 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:54:34,278 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Initializing shard [people] 08:54:34,278 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-config_testChangeMemberVotingStatusForShard is initialized 08:54:34,279 AM [cluster-test-shard-dispatcher-11] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:54:34,279 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,279 AM [cluster-test-shard-dispatcher-20] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,279 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:54:34,279 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,280 AM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,280 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 08:54:34,280 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 08:54:34,282 AM [cluster-test-shard-dispatcher-19] [INFO] PekkoRecovery - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,282 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,282 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,282 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:54:34,282 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [people] 08:54:34,282 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 08:54:34,282 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 08:54:34,282 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 08:54:34,282 AM [cluster-test-shard-dispatcher-22] [INFO] PekkoRecovery - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,283 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,283 AM [cluster-test-shard-dispatcher-22] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,283 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:54:34,283 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-oper-member-1: Initializing shard [cars] 08:54:34,283 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 08:54:34,283 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 08:54:34,283 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 08:54:34,285 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,285 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,285 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:54:34,285 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 08:54:34,286 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 08:54:34,288 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 08:54:34,299 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2559] with UID [3658559936440253430] 08:54:34,299 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Starting up, Pekko version [1.0.3] ... 08:54:34,301 AM [cluster-test-pekko.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:34,301 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Started up successfully 08:54:34,304 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:34,304 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:34,318 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,323 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,326 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 08:54:34,335 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1219584191] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,336 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-2 08:54:34,336 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: snapshot not present 08:54:34,337 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-2: Recovery complete 08:54:34,338 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,338 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: Creating local shard: member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,340 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:54:34,341 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 08:54:34,342 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,343 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,345 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@c4450b6} 08:54:34,346 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:34,346 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,346 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,346 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:54:34,347 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:54:34,347 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:54:34,347 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1211877294 created and ready for shard:member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,347 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:54:34,348 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 08:54:34,353 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:54:34,354 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-2 08:54:34,354 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-2: snapshot not present 08:54:34,355 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:54:34,355 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#600772224 created and ready for shard:member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,355 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#240887551] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,355 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard#438463313] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,355 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1211877294] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,355 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#854965456] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,355 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#600772224] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,356 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,361 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,362 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,362 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,362 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:54:34,362 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [people] 08:54:34,362 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-people-config_testChangeMemberVotingStatusForShard is initialized 08:54:34,362 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 08:54:34,363 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Recovery complete 08:54:34,363 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,364 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,364 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Creating local shard: member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,363 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 08:54:34,366 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,366 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RecoveringClientActorBehavior - member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 08:54:34,366 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,366 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,366 AM [cluster-test-pekko.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:54:34,366 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-config-member-2: Initializing shard [cars] 08:54:34,366 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 08:54:34,366 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 08:54:34,366 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received role changed for member-2-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 08:54:34,366 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 08:54:34,368 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@295299eb} 08:54:34,368 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:34,368 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,369 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,370 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard#1416609882] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,370 AM [cluster-test-shard-dispatcher-27] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:54:34,371 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-1706228781 created and ready for shard:member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,371 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-1706228781] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,371 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:54:34,371 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard#-239126646] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,372 AM [cluster-test-pekko.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-181849387 created and ready for shard:member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,372 AM [cluster-test-shard-dispatcher-14] [INFO] RaftActor - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:54:34,372 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-181849387] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,373 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:54:34,374 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,374 AM [cluster-test-shard-dispatcher-27] [INFO] RaftActor - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:54:34,375 AM [cluster-test-shard-dispatcher-27] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:54:34,376 AM [cluster-test-shard-dispatcher-14] [INFO] PekkoRecovery - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,376 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [people] 08:54:34,376 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 08:54:34,376 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,376 AM [cluster-test-shard-dispatcher-14] [DEBUG] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,376 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:54:34,377 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,377 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 08:54:34,377 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 08:54:34,379 AM [cluster-test-shard-dispatcher-25] [INFO] PekkoRecovery - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,379 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-oper-member-2: Initializing shard [cars] 08:54:34,379 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 08:54:34,379 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,379 AM [cluster-test-shard-dispatcher-25] [DEBUG] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,379 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:54:34,380 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 08:54:34,380 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received role changed for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 08:54:34,386 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 08:54:34,392 AM [cluster-test-pekko.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2559] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:54:34,393 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:34,393 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:34,393 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,393 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,393 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,393 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,393 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,393 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,393 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,393 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,393 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,393 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,393 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,393 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,404 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 08:54:34,413 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [pekko://cluster-test@127.0.0.1:2557] with UID [-5198626625314572785] 08:54:34,414 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Starting up, Pekko version [1.0.3] ... 08:54:34,416 AM [cluster-test-pekko.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=pekko:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:34,416 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Started up successfully 08:54:34,416 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#downing 08:54:34,417 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://pekko.apache.org/docs/pekko/current/typed/cluster.html#joining 08:54:34,425 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,427 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,429 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-config_testChangeMemberVotingStatusForShard 08:54:34,436 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-940141458] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,444 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:54:34,445 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-config-member-3 08:54:34,445 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: snapshot not present 08:54:34,445 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} started 08:54:34,446 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,447 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 08:54:34,454 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-3: Recovery complete 08:54:34,454 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,455 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-oper_testChangeMemberVotingStatusForShard 08:54:34,455 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Creating local shard: member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,455 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@67e0bd5a} 08:54:34,455 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:34,455 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,456 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard#-895273954] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,456 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-oper-member-3 08:54:34,456 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-oper-member-3: snapshot not present 08:54:34,456 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:54:34,457 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-666574052] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,457 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1596007755 created and ready for shard:member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,457 AM [cluster-test-shard-dispatcher-12] [INFO] RaftActor - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:54:34,457 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard-notifier#1596007755] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,458 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:54:34,459 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,459 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:54:34,460 AM [cluster-test-pekko.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1079772529 created and ready for shard:member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,460 AM [cluster-test-shard-dispatcher-17] [INFO] RaftActor - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:54:34,460 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard#-2025582488] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,460 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard-notifier#-1079772529] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,463 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Node [pekko://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 08:54:34,463 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,464 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RecoveringClientActorBehavior - member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard: saving tombstone ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 08:54:34,465 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} started 08:54:34,465 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:54:34,465 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:54:34,465 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,466 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [cars] 08:54:34,466 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard is initialized 08:54:34,466 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,466 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,466 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:54:34,466 AM [cluster-test-pekko.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 08:54:34,466 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-cars-config_testChangeMemberVotingStatusForShard from null to Follower 08:54:34,466 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,467 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Recovery complete 08:54:34,467 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,468 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Creating local shard: member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,468 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: onRegisterForShardAvailabilityChanges: RegisterForShardAvailabilityChanges{callback=org.opendaylight.controller.cluster.databroker.actors.dds.ModuleShardBackendResolver$$Lambda/0x00007fccc0774a08@42506bc9} 08:54:34,468 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 08:54:34,468 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,468 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:34,468 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,468 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,468 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,468 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,468 AM [cluster-test-shard-dispatcher-32] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:34,468 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,469 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-1}, address: pekko://cluster-test@127.0.0.1:2558 08:54:34,469 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,469 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,469 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,469 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,469 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:54:34,469 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-1041011932 created and ready for shard:member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,469 AM [cluster-test-shard-dispatcher-28] [INFO] RaftActor - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:54:34,470 AM [cluster-test-shard-dispatcher-12] [INFO] PekkoRecovery - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,470 AM [cluster-test-shard-dispatcher-32] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Shard created, persistent : true 08:54:34,470 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-609259541 created and ready for shard:member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,470 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - Welcome from [pekko://cluster-test@127.0.0.1:2558] 08:54:34,470 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard#118448117] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,470 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard-notifier#-1041011932] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,470 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard#-1314121718] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,470 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - pekkoDeadLetter Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard-notifier#-609259541] to Actor[pekko://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[pekko://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'pekko.log-dead-letters' and 'pekko.log-dead-letters-during-shutdown'. 08:54:34,470 AM [cluster-test-shard-dispatcher-32] [INFO] RaftActor - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Starting recovery with journal batch size 1 08:54:34,470 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-3: Initializing shard [people] 08:54:34,470 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-3-shard-people-config_testChangeMemberVotingStatusForShard is initialized 08:54:34,470 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:54:34,471 AM [cluster-test-shard-dispatcher-32] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: journal open: applyTo=0 08:54:34,471 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,471 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 08:54:34,473 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,473 AM [cluster-test-shard-dispatcher-12] [DEBUG] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,473 AM [cluster-test-shard-dispatcher-12] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:34,473 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:54:34,473 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-config_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard 08:54:34,473 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received role changed for member-3-shard-people-config_testChangeMemberVotingStatusForShard from null to Follower 08:54:34,473 AM [cluster-test-shard-dispatcher-28] [INFO] PekkoRecovery - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,474 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [cars] 08:54:34,474 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard is initialized 08:54:34,474 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,474 AM [cluster-test-shard-dispatcher-28] [DEBUG] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,474 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:54:34,474 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:34,474 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 08:54:34,474 AM [cluster-test-shard-dispatcher-17] [INFO] PekkoRecovery - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Local TermInfo store seeded with TermInfo{term=0} 08:54:34,474 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-cars-oper_testChangeMemberVotingStatusForShard from null to Follower 08:54:34,474 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-oper-member-3: Initializing shard [people] 08:54:34,474 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard is initialized 08:54:34,474 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,474 AM [cluster-test-shard-dispatcher-17] [DEBUG] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,474 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from null to Follower 08:54:34,474 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-1-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2558/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:34,474 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-oper_testChangeMemberVotingStatusForShard , registered listener pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard 08:54:34,476 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3: Received role changed for member-3-shard-people-oper_testChangeMemberVotingStatusForShard from null to Follower 08:54:34,639 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 08:54:34,639 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:34,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,639 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 08:54:34,639 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Follower to Candidate 08:54:34,647 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 08:54:34,647 AM [cluster-test-shard-dispatcher-21] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:34,647 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,647 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,647 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 08:54:34,647 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 08:54:34,648 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 08:54:34,648 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:34,648 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,648 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,648 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 08:54:34,648 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Follower to Candidate 08:54:34,698 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 1 08:54:34,698 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 08:54:34,698 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: false, hasLeader : false 08:54:34,698 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onStateChanged: Closing all transaction chains because shard is no longer the leader 08:54:34,698 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Follower to Candidate 08:54:34,698 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Follower to Candidate 08:54:34,977 AM [cluster-test-shard-dispatcher-21] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 08:54:34,977 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 08:54:35,28 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 08:54:35,108 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 2 08:54:35,247 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:35,247 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:35,247 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2557] to [Up] 08:54:35,247 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - pekkoMemberChanged Cluster Node [pekko://cluster-test@127.0.0.1:2558] - Leader is moving node [pekko://cluster-test@127.0.0.1:2559] to [Up] 08:54:35,247 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:35,247 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-oper-member-1: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:35,247 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-24] [INFO] PeerInfos - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,247 AM [cluster-test-shard-dispatcher-21] [INFO] PeerInfos - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,248 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,248 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,248 AM [cluster-test-shard-dispatcher-19] [INFO] PeerInfos - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,252 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:35,252 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,252 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,252 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,252 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:35,253 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-14] [INFO] PeerInfos - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-oper-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:35,253 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-2: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:35,253 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-25] [INFO] PeerInfos - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,253 AM [cluster-test-shard-dispatcher-27] [INFO] PeerInfos - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,297 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 08:54:35,298 AM [cluster-test-shard-dispatcher-17] [INFO] Follower - member-3-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:35,299 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:35,308 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 08:54:35,308 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-people-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:54:35,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 08:54:35,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 08:54:35,308 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 08:54:35,308 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@28e2a5ab 08:54:35,308 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-people-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 08:54:35,310 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 08:54:35,310 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:35,310 AM [cluster-test-shard-dispatcher-17] [INFO] Follower - member-3-shard-people-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-people-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:35,310 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@57478682 08:54:35,310 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2f1abc11 08:54:35,310 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-oper-member-3 Received follower initial sync status for member-3-shard-people-oper_testChangeMemberVotingStatusForShard status sync done false 08:54:35,317 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-people-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:54:35,317 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 08:54:35,317 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 08:54:35,317 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 08:54:35,317 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@26255ec7 08:54:35,317 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-people-config_testChangeMemberVotingStatusForShard from Candidate to Leader 08:54:35,318 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@4bdaa9b3 08:54:35,318 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 08:54:35,325 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@18139b86 08:54:35,325 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-people-config_testChangeMemberVotingStatusForShard status sync done false 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-3}, address: pekko://cluster-test@127.0.0.1:2557 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-oper-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:35,334 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2557/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-config-member-3: Received MemberUp: memberName: MemberName{name=member-2}, address: pekko://cluster-test@127.0.0.1:2559 08:54:35,334 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard with address pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard to pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-oper_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-oper_testChangeMemberVotingStatusForShard/member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-28] [INFO] PeerInfos - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-cars-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,334 AM [cluster-test-shard-dispatcher-17] [INFO] PeerInfos - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Peer address for peer member-2-shard-people-config_testChangeMemberVotingStatusForShard set to pekko://cluster-test@127.0.0.1:2559/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,428 AM [cluster-test-shard-dispatcher-23] [INFO] Candidate - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 08:54:35,430 AM [cluster-test-shard-dispatcher-14] [INFO] Follower - member-2-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:35,430 AM [cluster-test-shard-dispatcher-33] [INFO] Follower - member-3-shard-cars-config_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:35,430 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 08:54:35,436 AM [cluster-test-shard-dispatcher-23] [INFO] RaftActorBehavior - member-1-shard-cars-config_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:54:35,436 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 08:54:35,437 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 08:54:35,437 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-config_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 08:54:35,437 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@233fd22c 08:54:35,437 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received role changed for member-1-shard-cars-config_testChangeMemberVotingStatusForShard from Candidate to Leader 08:54:35,437 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 08:54:35,437 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 08:54:35,437 AM [main] [INFO] AbstractDataStore - Beginning to wait for data store to become ready : ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:54:35,438 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 08:54:35,438 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done false 08:54:35,438 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@77f7492a 08:54:35,438 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@40fd3689 08:54:35,438 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-config-member-2: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 08:54:35,438 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-config-member-3: All Shards are ready - data store config_testChangeMemberVotingStatusForShard is ready 08:54:35,438 AM [main] [DEBUG] AbstractDataStore - Data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} is now ready 08:54:35,438 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-631104172] 08:54:35,438 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:35,438 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#183280174] 08:54:35,438 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#183280174] 08:54:35,439 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1219584191] 08:54:35,439 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:54:35,439 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#240887551] 08:54:35,439 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#240887551] 08:54:35,439 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-940141458] 08:54:35,439 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:54:35,439 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-666574052] 08:54:35,439 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-666574052] 08:54:35,439 AM [main] [INFO] ClusterAdminRpcService - Change member voting states for shard ShardName{value=cars}: {member-2=false, member-3=false} 08:54:35,440 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: onChangeShardServersVotingStatus: ChangeShardMembersVotingStatus [shardName=cars, meberVotingStatusMap={member-2=false, member-3=false}] 08:54:35,440 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:54:35,440 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: Sending ChangeServersVotingStatus message ChangeServersVotingStatus [serverVotingStatusMap={member-2-shard-cars-config_testChangeMemberVotingStatusForShard=false, member-3-shard-cars-config_testChangeMemberVotingStatusForShard=false}, serversVisited=[]] to local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,442 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorVotingConfigSupport - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: VotingConfig{serverInfo=[ServerInfo[peerId=member-2-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-3-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=false], ServerInfo[peerId=member-1-shard-cars-config_testChangeMemberVotingStatusForShard, isVoting=true]]} has been successfully replicated to a majority of followers 08:54:35,444 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: Received ServerChangeReply{status=OK, leaderHint=member-1-shard-cars-config_testChangeMemberVotingStatusForShard} from local shard pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,444 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: ChangeServersVotingStatus succeeded for shard cars 08:54:35,444 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] ClusterAdminRpcService - Successfully changed member voting states for shard ShardName{value=cars} 08:54:35,446 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-631104172] 08:54:35,446 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-config-member-1: findLocalShard : cars 08:54:35,446 AM [cluster-test-pekko.actor.default-dispatcher-17] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#183280174] 08:54:35,446 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-1-shard-cars-config_testChangeMemberVotingStatusForShard#183280174] 08:54:35,447 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-1219584191] 08:54:35,447 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-config-member-2: findLocalShard : cars 08:54:35,447 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#240887551] 08:54:35,447 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-2-shard-cars-config_testChangeMemberVotingStatusForShard#240887551] 08:54:35,449 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-config-member-3 Received follower initial sync status for member-3-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 08:54:35,449 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard#-940141458] 08:54:35,449 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-config-member-2 Received follower initial sync status for member-2-shard-cars-config_testChangeMemberVotingStatusForShard status sync done true 08:54:35,449 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-config-member-3: findLocalShard : cars 08:54:35,449 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-666574052] 08:54:35,449 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[pekko://cluster-test/user/shardmanager-config_testChangeMemberVotingStatusForShard/member-3-shard-cars-config_testChangeMemberVotingStatusForShard#-666574052] 08:54:35,449 AM [cluster-test-shard-dispatcher-19] [INFO] Candidate - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate): Starting new election term 3 08:54:35,449 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:54:35,450 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,450 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-config-member-3: Issuing gracefulStop to shard member-3-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,450 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-3 - waiting on 2 shards 08:54:35,450 AM [cluster-test-shard-dispatcher-17] [INFO] Shard - member-3-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 08:54:35,450 AM [cluster-test-shard-dispatcher-12] [INFO] Follower - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:35,451 AM [cluster-test-shard-dispatcher-33] [INFO] Shard - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 08:54:35,451 AM [cluster-test-shard-dispatcher-29] [INFO] Follower - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard (Follower): Term 3 in "RequestVote{term=3, candidateId=member-1-shard-cars-oper_testChangeMemberVotingStatusForShard, lastLogIndex=-1, lastLogTerm=-1}" message is greater than follower's term 0 - updating term 08:54:35,451 AM [cluster-test-shard-dispatcher-33] [INFO] EnabledRaftStorage - member-3-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 08:54:35,451 AM [cluster-test-shard-dispatcher-17] [INFO] EnabledRaftStorage - member-3-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 08:54:35,452 AM [cluster-test-pekko.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-config-member-3: All shards shutdown - sending PoisonPill to self 08:54:35,452 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-3 08:54:35,452 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 08:54:35,452 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,452 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-oper-member-3: Issuing gracefulStop to shard member-3-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,452 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-3 - waiting on 2 shards 08:54:35,452 AM [cluster-test-shard-dispatcher-28] [INFO] Shard - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 08:54:35,453 AM [cluster-test-shard-dispatcher-28] [INFO] EnabledRaftStorage - member-3-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 08:54:35,456 AM [cluster-test-shard-dispatcher-12] [INFO] Shard - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 08:54:35,456 AM [cluster-test-shard-dispatcher-12] [INFO] EnabledRaftStorage - member-3-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 08:54:35,456 AM [cluster-test-pekko.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-oper-member-3: All shards shutdown - sending PoisonPill to self 08:54:35,456 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-3 08:54:35,457 AM [cluster-test-shard-dispatcher-19] [INFO] RaftActorBehavior - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 08:54:35,457 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: onLeadershipChange, isLeader: true, hasLeader : true 08:54:35,457 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: became leader with frontend state for [] 08:54:35,457 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard , received role change from Candidate to Leader 08:54:35,458 AM [cluster-test-pekko.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:35,458 AM [cluster-test-pekko.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:35,458 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@2bd59d31 08:54:35,458 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@221acedb 08:54:35,458 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 08:54:35,458 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-oper-member-2 Received follower initial sync status for member-2-shard-cars-oper_testChangeMemberVotingStatusForShard status sync done false 08:54:35,458 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received role changed for member-1-shard-cars-oper_testChangeMemberVotingStatusForShard from Candidate to Leader 08:54:35,458 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: All Shards are ready - data store oper_testChangeMemberVotingStatusForShard is ready 08:54:35,462 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:35,462 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:35,462 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:35,462 AM [cluster-test-pekko.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:35,463 AM [cluster-test-pekko.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 08:54:35,470 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:54:35,471 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:35,471 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-config-member-2: Issuing gracefulStop to shard member-2-shard-people-config_testChangeMemberVotingStatusForShard 08:54:35,471 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-2 - waiting on 2 shards 08:54:35,471 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 08:54:35,471 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 08:54:35,472 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 08:54:35,472 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 08:54:35,473 AM [cluster-test-pekko.actor.default-dispatcher-7] [DEBUG] ShardManager - shard-manager-config-member-2: All shards shutdown - sending PoisonPill to self 08:54:35,473 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-2 08:54:35,473 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 08:54:35,474 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:35,474 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-oper-member-2: Issuing gracefulStop to shard member-2-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:35,474 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-2 - waiting on 2 shards 08:54:35,474 AM [cluster-test-shard-dispatcher-25] [INFO] Shard - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 08:54:35,474 AM [cluster-test-shard-dispatcher-14] [INFO] Shard - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 08:54:35,475 AM [cluster-test-shard-dispatcher-25] [INFO] EnabledRaftStorage - member-2-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 08:54:35,475 AM [cluster-test-shard-dispatcher-14] [INFO] EnabledRaftStorage - member-2-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 08:54:35,475 AM [cluster-test-pekko.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-oper-member-2: All shards shutdown - sending PoisonPill to self 08:54:35,475 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-2 08:54:35,476 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:35,476 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:36,265 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [pekko://cluster-test@127.0.0.1:2558] - is no longer leader 08:54:36,475 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2557] having UID [-5198626625314572785] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:36,491 AM [cluster-test-pekko.actor.default-dispatcher-7] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:36,491 AM [cluster-test-pekko.actor.default-dispatcher-7] [WARN] Materializer - [outbound connection to [pekko://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 08:54:36,493 AM [cluster-test-pekko.actor.default-dispatcher-7] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:54:36.494] [cluster-test-pekko.actor.internal-dispatcher-10] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 08:54:36,495 AM [cluster-test-pekko.actor.default-dispatcher-17] [INFO] Association - Association to [pekko://cluster-test@127.0.0.1:2559] having UID [3658559936440253430] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 08:54:36,498 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-config_testChangeMemberVotingStatusForShard, generation=0} 08:54:36,498 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-cars-config_testChangeMemberVotingStatusForShard 08:54:36,498 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-config-member-1: Issuing gracefulStop to shard member-1-shard-people-config_testChangeMemberVotingStatusForShard 08:54:36,498 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-config-member-1 - waiting on 2 shards 08:54:36,499 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@671854fc 08:54:36,499 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@558998b5 08:54:36,499 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@671854fc 08:54:36,499 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@558998b5 08:54:36,499 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 08:54:36,499 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 08:54:36,499 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@33242c55 08:54:36,499 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-config-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@567de0d4 08:54:36,915 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: leader transfer aborted 08:54:36,915 AM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 416.2 ms 08:54:36,915 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: In unpauseLeader 08:54:36,916 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-config_testChangeMemberVotingStatusForShard: Stopping Shard 08:54:36,916 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-people-config_testChangeMemberVotingStatusForShard: journal closed 08:54:37,35 AM [cluster-test-shard-dispatcher-19] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: leader transfer aborted 08:54:37,35 AM [cluster-test-shard-dispatcher-19] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 536.4 ms 08:54:37,35 AM [cluster-test-shard-dispatcher-19] [DEBUG] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: In unpauseLeader 08:54:37,35 AM [cluster-test-shard-dispatcher-19] [INFO] Shard - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: Stopping Shard 08:54:37,36 AM [cluster-test-shard-dispatcher-19] [INFO] EnabledRaftStorage - member-1-shard-cars-config_testChangeMemberVotingStatusForShard: journal closed 08:54:37,36 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-config-member-1: All shards shutdown - sending PoisonPill to self 08:54:37,36 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-config-member-1 08:54:37,37 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-oper_testChangeMemberVotingStatusForShard, generation=0} 08:54:37,37 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-cars-oper_testChangeMemberVotingStatusForShard 08:54:37,37 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-oper-member-1: Issuing gracefulStop to shard member-1-shard-people-oper_testChangeMemberVotingStatusForShard 08:54:37,37 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-oper-member-1 - waiting on 2 shards 08:54:37,37 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@af20c9a 08:54:37,37 AM [cluster-test-shard-dispatcher-23] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5aea7a49 08:54:37,37 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5aea7a49 08:54:37,37 AM [cluster-test-shard-dispatcher-23] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 08:54:37,38 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-oper-member-1: Received LeaderStateChanged message: org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged@56b5d4b5 08:54:37,38 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@9efec87 08:54:37,38 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@9efec87 08:54:37,38 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: pauseLeader successfully completed - doing transfer 08:54:37,375 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 08:54:37,375 AM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 337.6 ms 08:54:37,375 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 08:54:37,376 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: Stopping Shard 08:54:37,376 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-cars-oper_testChangeMemberVotingStatusForShard: journal closed 08:54:37,545 AM [cluster-test-shard-dispatcher-21] [DEBUG] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: leader transfer aborted 08:54:37,545 AM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Failed to transfer leadership in 507.8 ms 08:54:37,545 AM [cluster-test-shard-dispatcher-21] [DEBUG] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: In unpauseLeader 08:54:37,545 AM [cluster-test-shard-dispatcher-21] [INFO] Shard - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: Stopping Shard 08:54:37,546 AM [cluster-test-shard-dispatcher-21] [INFO] EnabledRaftStorage - member-1-shard-people-oper_testChangeMemberVotingStatusForShard: journal closed 08:54:37,546 AM [cluster-test-pekko.actor.default-dispatcher-18] [DEBUG] ShardManager - shard-manager-oper-member-1: All shards shutdown - sending PoisonPill to self 08:54:37,546 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-oper-member-1 08:54:37,547 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 08:54:37,547 AM [cluster-test-pekko.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 08:54:37,557 AM [cluster-test-pekko.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/10/2026 08:54:37.558] [cluster-test-pekko.actor.internal-dispatcher-4] [Cluster(pekko://cluster-test)] Could not unregister Cluster JMX MBean with name=pekko:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'pekko.cluster.jmx.multi-mbeans-in-same-jvm = on' in config